diff --git a/NOTIFICATIONS.md b/NOTIFICATIONS.md new file mode 100644 index 0000000000..b50b399571 --- /dev/null +++ b/NOTIFICATIONS.md @@ -0,0 +1,108 @@ +# Notifications + +This explains how notifications work in OnTrack after the unification. + +## The idea + +Before, each type of message did its own thing. Emails were sent from many +places. There was no single system. + +Now there is one system. You send a notification once. It goes out on all the +channels the user has turned on: in-app, email, and Web Push when the deployment +has VAPID keys and the user has subscribed a browser. + +## The flow + + something happens in the app + -> you call NotificationService.notify(...) + -> saves an in-app notification (the bell) + -> queues an ID-only email job + -> queues an ID-only push job + -> Sidekiq workers reload the notification and contact providers + +You only call one thing. The system handles the rest. + +## How to send one + +Call this from anywhere in the API code: + + NotificationService.notify( + user: project.student, + type: 'feedback', + event: 'task_comment_created', + message: "New feedback is ready for #{task_definition.name}.", + link: "/#/projects/#{project.id}" + ) + +- user: who gets it. +- type: the category. One of task, feedback, portfolio, extension, general. + This is what the user's on/off setting controls. +- event: the specific thing that happened, as a lower_snake_case string. + Required. Use one event name per ticket, and use the same name every time you + raise that notification, so a notification can always be traced back to the + code that sent it. +- message: the text the user sees. Keep it short, 500 characters at most. +- link: where clicking it should take them. Optional. + +type and event are different on purpose. type is the coarse category the user +switches off in their profile. event is the fine-grained reason, and there will +be many events inside one type. + +## Types and preferences + +Each user already has three on/off settings in their profile: + +- receive_task_notifications +- receive_feedback_notifications +- receive_portfolio_notifications + +The type you pass maps to one of these settings. + +- task uses receive_task_notifications +- feedback uses receive_feedback_notifications +- portfolio uses receive_portfolio_notifications +- extension and general are always sent + +If the matching setting is off, nothing is sent. Not the bell, not the email, +not the push. One switch controls all channels. This keeps it simple. We can add +per-channel switches later if we want. + +## The pieces + +- app/models/notification.rb: the notification record. Has the type, message, + link, and whether it has been read. +- app/services/notification_service.rb: the one entry point. Checks the setting, + saves the record, and queues the email and push channel jobs. +- app/sidekiq/notification_email_job.rb: reloads a notification by id and sends + its email on the `mailers` queue. +- app/sidekiq/push_notification_delivery_job.rb: reloads a notification by id + and hands it to the Web Push delivery channel on the `notifications` queue. +- app/services/push_notification_service.rb: the Web Push delivery channel. It + remains a safe no-op until both VAPID keys are configured. +- app/mailers/notifications_mailer.rb: the email. New method single_notification + with templates in app/views/notifications_mailer. +- app/api/notifications_api.rb: the endpoints the web app calls. +- app/api/entities/notification_entity.rb: the shape of the data sent back. + +## The endpoints + + GET /api/notifications list my notifications + GET /api/notifications/unread_count how many I have not read + PUT /api/notifications/:id/read mark one as read + PUT /api/notifications/read_all mark all as read + DELETE /api/notifications/:id delete one + + GET /api/push_subscriptions list my browser subscriptions + POST /api/push_subscriptions register or update a browser + DELETE /api/push_subscriptions remove the browser identified by endpoint + +Every endpoint only ever touches the current user's own notifications. + +## What is on now + +- In-app: working. The record is saved and the endpoints return it. +- Email: working. Best effort. If email fails, the in-app notification is still + saved. +- Push: implemented and deliberately configuration-gated. It sends only when + VAPID keys are set and that user has opted in from a supported browser. Keep + the production keys blank until browser/device acceptance testing is complete. diff --git a/app/api/additional_notification_email_verification_api.rb b/app/api/additional_notification_email_verification_api.rb new file mode 100644 index 0000000000..0a4021bbc4 --- /dev/null +++ b/app/api/additional_notification_email_verification_api.rb @@ -0,0 +1,17 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailVerificationApi < Grape::API + desc 'Verify ownership of an additional notification email' + params do + requires :token, type: String + end + post '/additional_notification_emails/verify' do + header 'Cache-Control', 'private, no-store' + AdditionalNotificationEmailService.verify(token: params[:token]) + { status: 'verified' } + rescue AdditionalNotificationEmailService::AlreadyVerified + error!({ error: 'This verification link has already been used.' }, 409) + rescue AdditionalNotificationEmailService::InvalidToken + error!({ error: 'This verification link is invalid or has expired.' }, 422) + end +end diff --git a/app/api/additional_notification_emails_api.rb b/app/api/additional_notification_emails_api.rb new file mode 100644 index 0000000000..3a5d3d0d9a --- /dev/null +++ b/app/api/additional_notification_emails_api.rb @@ -0,0 +1,77 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailsApi < Grape::API + helpers AuthenticationHelpers + + before do + authenticated? + header 'Cache-Control', 'private, no-store' + end + + helpers do + def own_user! + error!({ error: 'You can only manage your own additional notification email.' }, 403) unless params[:user_id] == current_user.id + + current_user + end + + def present_additional_email(record) + if record.nil? + { status: 'none', email: nil, verification_expires_at: nil } + else + { + status: record.verified? ? 'verified' : 'pending', + email: record.email, + verification_expires_at: record.verification_expires_at + } + end + end + end + + desc 'Get the current user additional notification email state' + params do + requires :user_id, type: Integer + end + get '/users/:user_id/additional_notification_email' do + user = own_user! + present present_additional_email(user.additional_notification_email) + end + + desc 'Request or replace an additional notification email' + params do + requires :user_id, type: Integer + requires :email, type: String + end + put '/users/:user_id/additional_notification_email' do + user = own_user! + record = AdditionalNotificationEmailService.request(user: user, email: params[:email]) + present present_additional_email(record) + rescue AdditionalNotificationEmailService::RateLimited + error!({ error: 'Too many verification requests. Try again in one hour.' }, 429) + end + + desc 'Resend additional notification email verification' + params do + requires :user_id, type: Integer + end + post '/users/:user_id/additional_notification_email/resend' do + user = own_user! + record = AdditionalNotificationEmailService.resend(user: user) + present present_additional_email(record) + rescue AdditionalNotificationEmailService::AlreadyVerified + error!({ error: 'This additional notification email is already verified.' }, 409) + rescue AdditionalNotificationEmailService::RateLimited + error!({ error: 'Too many verification requests. Try again in one hour.' }, 429) + end + + desc 'Remove the current user additional notification email' + params do + requires :user_id, type: Integer + end + delete '/users/:user_id/additional_notification_email' do + user = own_user! + AdditionalNotificationEmailService.remove(user: user) + status 204 + body false + end +end diff --git a/app/api/entities/notification_entity.rb b/app/api/entities/notification_entity.rb new file mode 100644 index 0000000000..227e6a50df --- /dev/null +++ b/app/api/entities/notification_entity.rb @@ -0,0 +1,19 @@ +module Entities + class NotificationEntity < Grape::Entity + expose :id + expose :notification_type + expose :event + expose :message + expose :link + expose :read_at + expose :created_at + + # Added after link, and never instead of it. A client that only knows link + # keeps working. A newer client opens the exact page from these ids, and a + # nil means the record is gone, so it can say so instead of opening a + # blank page. + Notification::TARGET_KEYS.each do |key| + expose(key) { |notification, _options| notification.target_ids[key] } + end + end +end diff --git a/app/api/entities/push_subscription_entity.rb b/app/api/entities/push_subscription_entity.rb new file mode 100644 index 0000000000..32a865022c --- /dev/null +++ b/app/api/entities/push_subscription_entity.rb @@ -0,0 +1,12 @@ +module Entities + class PushSubscriptionEntity < Grape::Entity + expose :id + expose :endpoint + expose :created_at + expose :updated_at + + # p256dh and auth are deliberately not exposed. They are the browser's own + # encryption material, the client already holds them, and nothing in the UI + # needs them read back. + end +end diff --git a/app/api/notifications_api.rb b/app/api/notifications_api.rb new file mode 100644 index 0000000000..2237c89102 --- /dev/null +++ b/app/api/notifications_api.rb @@ -0,0 +1,80 @@ +require 'grape' + +class NotificationsApi < Grape::API + helpers AuthenticationHelpers + helpers AuthorisationHelpers + + before do + authenticated? + end + + desc 'Get the current user notifications' + params do + optional :unread_only, type: Boolean, default: false, desc: 'Only return unread notifications' + end + get '/notifications' do + notifications = current_user.notifications.recent_first + notifications = notifications.unread if params[:unread_only] + + present notifications, with: Entities::NotificationEntity + end + + desc 'Get the current user unread notification count' + get '/notifications/unread_count' do + { count: current_user.notifications.unread.count } + end + + desc 'Mark a notification as read' + params do + requires :id, type: Integer, desc: 'The notification id' + end + put '/notifications/:id/read' do + notification = current_user.notifications.find(params[:id]) + notification.mark_read! + + present notification, with: Entities::NotificationEntity + end + + desc 'Mark all of the current user notifications as read' + put '/notifications/read_all' do + # rubocop:disable Rails/SkipsModelValidations + current_user.notifications.unread.update_all(read_at: Time.zone.now) + # rubocop:enable Rails/SkipsModelValidations + + status 200 + { success: true } + end + + desc 'Delete the current user notifications up to a confirmed boundary' + params do + requires :through_id, + type: Integer, + values: ->(value) { value.positive? }, + desc: 'Delete only notifications whose id is at or below this value' + end + delete '/notifications' do + # The boundary is part of the contract, not an optimisation. A notification + # can arrive after the browser has shown "Delete all" and while the user is + # reading the confirmation. Deleting the unbounded association here would + # remove that unseen notification too. Scoping through current_user keeps + # the operation account-local in the same way as the single-row endpoint. + deleted_count = current_user.notifications + .where('notifications.id <= ?', params[:through_id]) + .delete_all + + status 200 + { success: true, deleted_count: deleted_count } + end + + desc 'Delete a notification' + params do + requires :id, type: Integer, desc: 'The notification id' + end + delete '/notifications/:id' do + notification = current_user.notifications.find(params[:id]) + notification.destroy! + + status 200 + { success: true } + end +end diff --git a/app/api/push_subscriptions_api.rb b/app/api/push_subscriptions_api.rb new file mode 100644 index 0000000000..b2c67158cc --- /dev/null +++ b/app/api/push_subscriptions_api.rb @@ -0,0 +1,64 @@ +require 'grape' + +class PushSubscriptionsApi < Grape::API + helpers AuthenticationHelpers + helpers AuthorisationHelpers + + before do + authenticated? + end + + desc 'Get the push subscriptions belonging to the current user' + get '/push_subscriptions' do + present current_user.push_subscriptions.order(:id), with: Entities::PushSubscriptionEntity + end + + # The endpoint posted here is later used as the target of an outbound request + # by PushNotificationService, so it is not accepted as free text. + # PushSubscription validates it against PUSH_SERVICE_HOSTS, and a URL that is + # not an https push service URL fails with a 400 from the handler in + # api_root.rb. PushNotificationService checks again before it sends. + desc 'Register this browser to receive push notifications' + params do + requires :endpoint, type: String, desc: 'The push service URL, from PushSubscription.endpoint' + requires :p256dh, type: String, desc: 'The browser public key, from PushSubscription.getKey("p256dh")' + requires :auth, type: String, desc: 'The browser auth secret, from PushSubscription.getKey("auth")' + end + post '/push_subscriptions' do + subscription = current_user.push_subscriptions.find_by(endpoint: params[:endpoint]) + + # Not ours, or not stored yet. An endpoint identifies a browser rather than + # a person, so an endpoint held by another user means someone has signed in + # on a machine that account used. Move the registration across instead of + # failing on the unique index. + # + # This is the only lookup in this file that is not scoped to current_user, + # and it is safe. The push service delivers to that browser no matter which + # row owns it, and the payload is encrypted to the keys posted here. Taking + # over someone else's endpoint cannot read their notifications, it can only + # stop them arriving, and you need the endpoint URL to try it at all. + subscription ||= PushSubscription.find_by(endpoint: params[:endpoint]) || PushSubscription.new + + subscription.assign_attributes( + user: current_user, + endpoint: params[:endpoint], + p256dh: params[:p256dh], + auth: params[:auth] + ) + subscription.save! + + present subscription, with: Entities::PushSubscriptionEntity + end + + desc 'Stop this browser receiving push notifications' + params do + requires :endpoint, type: String, desc: 'The push service URL to remove' + end + delete '/push_subscriptions' do + subscription = current_user.push_subscriptions.find_by!(endpoint: params[:endpoint]) + subscription.destroy! + + status 200 + { success: true } + end +end diff --git a/app/api/task_definitions_api.rb b/app/api/task_definitions_api.rb index 5e0be83ee7..4c1657fb2d 100644 --- a/app/api/task_definitions_api.rb +++ b/app/api/task_definitions_api.rb @@ -40,7 +40,8 @@ class TaskDefinitionsApi < Grape::API optional :scorm_time_delay_enabled, type: Boolean, desc: 'Whether there is an incremental time delay between SCORM test attempts' optional :scorm_attempt_limit, type: Integer, desc: 'The number of times a SCORM test can be attempted' optional :assess_in_portfolio_only, type: Boolean, desc: 'Whether a task can only be signed off during portfolio assessment' - optional :requires_discussion, type: Boolean, desc: 'Whether task must be discussed in class before it can be signed off as complete' + optional :resubmission_extensions_enabled, type: Boolean, desc: 'Allow the unit resubmission extension for future feedback on this task' + optional :requires_discussion, type: Boolean, desc: 'Whether task must be discussed in class before it can be signed off as complete' optional :use_resources_for_jplag_base_code, type: Boolean, desc: 'Include the common base code from task resources for JPlag comparisons' optional :lock_assessments_to_tutorial_stream, type: Boolean, desc: 'Only allow tutors in this tutorial stream to assess this task' end @@ -77,6 +78,7 @@ class TaskDefinitionsApi < Grape::API :similarity_language, :assess_in_portfolio_only, :requires_discussion, + :resubmission_extensions_enabled, :upload_requirements, :unit_id, :use_resources_for_jplag_base_code, @@ -91,6 +93,10 @@ class TaskDefinitionsApi < Grape::API end task_def = TaskDefinition.new(task_params) + if task_params.key?(:resubmission_extensions_enabled) + task_def.resubmission_extensions_changed_by_id = current_user.id + task_def.resubmission_extensions_changed_at = Time.current + end # Set the tutorial stream tutorial_stream_abbr = params[:task_def][:tutorial_stream_abbr] @@ -108,8 +114,11 @@ class TaskDefinitionsApi < Grape::API end task_def.save! + NewTaskAvailableNotificationJob.track_and_enqueue(task_def) - present task_def, with: Entities::TaskDefinitionEntity, my_role: unit.role_for(current_user) + present task_def, + with: Entities::TaskDefinitionEntity, + my_role: unit.role_for(current_user) end desc 'Edits the given task definition' @@ -140,7 +149,8 @@ class TaskDefinitionsApi < Grape::API optional :overseer_image_id, type: Integer, desc: 'The id of the Docker image name for overseer' optional :similarity_language, type: String, desc: 'The language to use for code similarity checks' optional :assess_in_portfolio_only, type: Boolean, desc: 'Whether a task can only be signed off during portfolio assessment' - optional :requires_discussion, type: Boolean, desc: 'Whether task must be discussed in class before it can be signed off as complete' + optional :resubmission_extensions_enabled, type: Boolean, desc: 'Allow the unit resubmission extension for future feedback on this task' + optional :requires_discussion, type: Boolean, desc: 'Whether task must be discussed in class before it can be signed off as complete' optional :use_resources_for_jplag_base_code, type: Boolean, desc: 'Include the common base code from task resources for JPlag comparisons' optional :lock_assessments_to_tutorial_stream, type: Boolean, desc: 'Only allow tutors in this tutorial stream to assess this task' optional :grade_due_dates, type: Array do @@ -186,6 +196,7 @@ class TaskDefinitionsApi < Grape::API :similarity_language, :assess_in_portfolio_only, :requires_discussion, + :resubmission_extensions_enabled, :upload_requirements, :use_resources_for_jplag_base_code, :lock_assessments_to_tutorial_stream @@ -211,8 +222,15 @@ class TaskDefinitionsApi < Grape::API error!({ error: 'Target grade is not enabled for this unit' }, 422) end + if task_params.key?(:resubmission_extensions_enabled) && + task_params[:resubmission_extensions_enabled] != task_def.resubmission_extensions_enabled + task_params[:resubmission_extensions_changed_by_id] = current_user.id + task_params[:resubmission_extensions_changed_at] = Time.current + end + # Bulk update task definition with permitted parameters task_def.update!(task_params) + due_date_change = task_def.saved_change_to_due_date # Set the tutorial stream tutorial_stream_abbr = params[:task_def][:tutorial_stream_abbr] @@ -266,6 +284,26 @@ class TaskDefinitionsApi < Grape::API end end + if due_date_change + previous_due_date, new_due_date = due_date_change.map do |value| + value&.to_date&.iso8601 + end + + begin + TaskDueDateChangedNotificationJob.perform_async( + task_def.id, + previous_due_date, + new_due_date, + SecureRandom.uuid + ) + rescue StandardError => e + Rails.logger.error( + "Failed to enqueue due-date notification for TaskDefinition " \ + "#{task_def.id}: #{e.class} - #{e.message}" + ) + end + end + present task_def, with: Entities::TaskDefinitionEntity, my_role: unit.role_for(current_user) end @@ -631,14 +669,14 @@ class TaskDefinitionsApi < Grape::API if task_def.has_task_sheet? path = task_def.task_sheet - filename = "#{task_def.unit.code}-#{task_def.abbreviation}.pdf" + filename = task_def.task_sheet_filename else path = Rails.root.join('public/resources/FileNotFound.pdf') filename = 'FileNotFound.pdf' end if params[:as_attachment] - header['Content-Disposition'] = "attachment; filename=#{filename}" + header['Content-Disposition'] = %(attachment; filename="#{filename}") end content_type 'application/pdf' diff --git a/app/helpers/notification_mail_helper.rb b/app/helpers/notification_mail_helper.rb new file mode 100644 index 0000000000..cbd3590017 --- /dev/null +++ b/app/helpers/notification_mail_helper.rb @@ -0,0 +1,148 @@ +# Style tokens for the notification mail layout. +# +# One map, so the badge at the top of a message, the rule above the card and the +# callout inside it all agree on the same colour. A template names a category and +# nothing else: it never repeats a hex value, and a template that forgets to name +# one still renders, in the neutral brand colour. +# +# Where an email is about a task status, it takes that status's own colour, so a +# status looks the same in the inbox as it does in the app. Where an email is not +# about a status, the colour still has to mean something: green for finished, red +# for broken, amber for a deadline, brand blue for neutral. No shade here is +# invented. +module NotificationMailHelper + BRAND = '#3939ff'.freeze + + # OnTrack's task status palette, copied from the web app's + # src/styles/tokens/_light.scss. Every one of these is designed to carry white + # text, which is what the badge draws on it. + STATUS_COLOURS = { + 'ready-for-feedback' => '#0078d5', + 'not-started' => '#757575', + 'working-on-it' => '#a86604', + 'need-help' => '#8366bc', + 'fix-and-resubmit' => '#8b750b', + 'feedback-exceeded' => '#ca4e33', + 'redo' => '#804000', + 'discuss' => '#20809c', + 'rediscuss' => '#126352', + 'demonstrate' => '#337ab7', + 'complete' => '#3b863b', + 'fail' => '#d93713', + 'time-exceeded' => '#d93713', + 'assess-in-portfolio' => '#8b750b', + 'attention-required' => '#cd4c10' + }.freeze + + # The glyph each status carries when a message is about that status alone. + STATUS_GLYPHS = { + 'ready-for-feedback' => { glyph: '✓', size: '24px' }, + 'not-started' => { glyph: '○', size: '22px' }, + 'working-on-it' => { glyph: '✎', size: '22px' }, + 'need-help' => { glyph: '?', size: '26px' }, + 'fix-and-resubmit' => { glyph: '↻', size: '24px' }, + 'feedback-exceeded' => { glyph: '▲', size: '20px' }, + 'redo' => { glyph: '↻', size: '24px' }, + 'discuss' => { glyph: '●●●', size: '11px' }, + 'rediscuss' => { glyph: '●●●', size: '11px' }, + 'demonstrate' => { glyph: '▶', size: '20px' }, + 'complete' => { glyph: '✓', size: '24px' }, + 'fail' => { glyph: '✗', size: '24px' }, + 'time-exceeded' => { glyph: '▲', size: '20px' }, + 'assess-in-portfolio' => { glyph: '▤', size: '22px' }, + 'attention-required' => { glyph: '!', size: '26px' } + }.freeze + + # Glyphs are written as HTML entities so the file stays ASCII and no mail + # client has to guess at an encoding. Each one was rendered on a coloured disc + # before it was chosen, which is how the clock and hourglass candidates were + # dropped: they default to emoji presentation and arrive as a colour picture + # sitting on a coloured circle. Everything kept is text presentation from a + # block the core fonts cover. + CATEGORIES = { + 'submission' => { glyph: '✓', size: '24px', status: 'complete' }, + 'ready_for_feedback' => { glyph: '✓', size: '24px', status: 'ready-for-feedback' }, + 'portfolio' => { glyph: '▤', size: '22px', status: 'complete' }, + 'feedback' => { glyph: '✎', size: '22px', accent: BRAND }, + 'comment' => { glyph: '●●●', size: '11px', accent: BRAND }, + 'discuss' => { glyph: '●●●', size: '11px', status: 'discuss' }, + 'help' => { glyph: '?', size: '26px', status: 'need-help' }, + 'announcement' => { glyph: '⚑', size: '24px', accent: BRAND }, + 'new' => { glyph: '★', size: '22px', accent: BRAND }, + 'session' => { glyph: '◷', size: '24px', accent: BRAND }, + 'change' => { glyph: '↻', size: '24px', accent: BRAND }, + 'extension' => { glyph: '⇥', size: '24px', accent: BRAND }, + 'group' => { glyph: '⁂', size: '24px', accent: BRAND }, + 'verification' => { glyph: '@', size: '22px', accent: BRAND }, + 'summary' => { glyph: '≡', size: '24px', accent: BRAND }, + 'due' => { glyph: '!', size: '26px', status: 'working-on-it' }, + 'overdue' => { glyph: '▲', size: '20px', status: 'time-exceeded' }, + 'alert' => { glyph: '✗', size: '24px', status: 'fail' } + }.freeze + + DEFAULT_CATEGORY = { glyph: '★', size: '22px', accent: BRAND }.freeze + + # Look up a category. Two forms are accepted: a category name, and + # "status:" for a message that carries whatever status a task is actually + # in. An unknown or blank name falls back to the neutral brand styling rather + # than raising, because a message that renders plainly is a far better failure + # than one that does not send at all. + def notification_mail_style(category) + key = category.to_s.strip + entry = + if key.start_with?('status:') + status_entry(key.delete_prefix('status:')) + else + CATEGORIES[key] + end + entry ||= DEFAULT_CATEGORY + + accent = entry[:accent] || STATUS_COLOURS[entry[:status]] || BRAND + { glyph: entry[:glyph], size: entry[:size], accent: accent, tint: notification_mail_tint(accent) } + end + + def notification_mail_accent(category) + notification_mail_style(category)[:accent] + end + + # Turn a task status into the name of a style. Task#status hands back a symbol + # such as :ready_for_feedback, while the web app's token for the same status is + # ready-for-feedback. + def notification_mail_status_category(status) + key = status.to_s.tr('_', '-') + STATUS_COLOURS.key?(key) ? "status:#{key}" : nil + end + + # The callout behind a colour: the same hue mixed most of the way to white. + # Derived rather than listed, so there is no second palette to drift. + def notification_mail_tint(accent) + notification_mail_mix(accent, 255, 0.08) || '#f5f5ff' + end + + # The same hue, darkened, for small text sitting on that tint. The status + # colours are built to carry white on a solid fill, which is a different job + # from carrying themselves on a pale wash: measured on the tint, the accents + # land near 4.1 against the 4.5 that 12px bold needs. Mixing a third of the + # way to black clears it without a second list of hexes to keep in step. + def notification_mail_text_accent(accent) + notification_mail_mix(accent, 0, 0.68) || '#1f2937' + end + + # Blend a hex colour towards `towards` (0 for black, 255 for white), keeping + # `weight` of the original. + def notification_mail_mix(accent, towards, weight) + hex = accent.to_s.delete_prefix('#') + return nil unless hex.length == 6 + + mixed = hex.scan(/../).map { |pair| ((pair.to_i(16) * weight) + (towards * (1 - weight))).round.clamp(0, 255) } + format('#%02x%02x%02x', *mixed) + end + + private + + def status_entry(key) + return nil unless STATUS_COLOURS.key?(key) + + STATUS_GLYPHS.fetch(key, { glyph: '✎', size: '22px' }).merge(status: key) + end +end diff --git a/app/mailers/additional_notification_email_mailer.rb b/app/mailers/additional_notification_email_mailer.rb new file mode 100644 index 0000000000..a3b3d37d2d --- /dev/null +++ b/app/mailers/additional_notification_email_mailer.rb @@ -0,0 +1,22 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailMailer < ApplicationMailer + layout "notification_mail" + + def verification(record) + @user = record.user + @product_name = Doubtfire::Application.config.institution[:product_name] + host = Doubtfire::Application.config.institution[:host].to_s.delete_suffix('/') + token_query = URI.encode_www_form(token: record.verification_token) + @verification_url = "#{host}/verify_additional_email##{token_query}" + + sender = Doubtfire::Application.config.institution[:email_sender].presence || + 'noreply@doubtfire.local' + + mail( + to: record.email, + subject: "Verify your additional #{@product_name} notification email", + **outbound_sender_headers(development_from: sender) + ) + end +end diff --git a/app/mailers/application_mailer.rb b/app/mailers/application_mailer.rb index ead50cd963..600efb9a0f 100644 --- a/app/mailers/application_mailer.rb +++ b/app/mailers/application_mailer.rb @@ -1,2 +1,39 @@ class ApplicationMailer < ActionMailer::Base + # Mailer views do not pick up app helpers on their own, and the notification + # layout needs the shared colour and glyph map for every message it renders. + helper :notification_mail + + private + + # Build a recipient or sender address through Mail, so a display name holding a + # quote or a comma cannot break out of the name and inject a second address, + # and strip control characters so a name cannot fold an extra header into the + # message. User#name comes from first_name/last_name, which are user-editable + # and validated for presence only, so the raw %("#{name}" <#{email}>) + # interpolation this replaces was header-injectable. + # + # It lives here rather than on one mailer because every mailer that addresses a + # person needs it, and the three that did not have it were the ones still + # interpolating by hand. + def address_with_name(user) + safe_name = user.name.to_s.gsub(/[[:cntrl:]]/, ' ').strip + address = Mail::Address.new(user.email.to_s) + address.display_name = safe_name + address.format + end + + # Azure Communication Services only accepts a verified sender in From. + # Keep the existing per-user From address outside production so local mail + # previews and development SMTP retain their current behaviour. In + # production, callers may preserve the human sender as Reply-To. + def outbound_sender_headers(development_from:, reply_to: nil) + return { from: development_from } unless Rails.env.production? + + configured_sender = Doubtfire::Application.config.institution[:email_sender].presence + raise ArgumentError, 'institution email_sender must be configured in production' if configured_sender.blank? + + headers = { from: configured_sender } + headers[:reply_to] = reply_to if reply_to.present? + headers + end end diff --git a/app/mailers/communications_mailer.rb b/app/mailers/communications_mailer.rb index 95034342d9..aa0f8deac5 100644 --- a/app/mailers/communications_mailer.rb +++ b/app/mailers/communications_mailer.rb @@ -1,4 +1,6 @@ class CommunicationsMailer < ApplicationMailer + layout "notification_mail" + def communication_email(to:, from:, subject:, body:, recipient:, sender:, unit:, rule:) @recipient = recipient @sender = sender @@ -11,7 +13,11 @@ def communication_email(to:, from:, subject:, body:, recipient:, sender:, unit:, @doubtfire_product_name = Doubtfire::Application.config.institution[:product_name] @unsubscribe_url = "#{@doubtfire_host}/edit_profile" - mail(to: to, from: from, subject: subject) + mail( + { to: to, subject: subject }.merge( + outbound_sender_headers(development_from: from, reply_to: from) + ) + ) end def action_log_email(payload) @@ -32,6 +38,10 @@ def action_log_email(payload) content: payload[:csv_content] } - mail(to: payload[:to], from: payload[:from], subject: payload[:subject]) + mail( + { to: payload[:to], subject: payload[:subject] }.merge( + outbound_sender_headers(development_from: payload[:from], reply_to: payload[:from]) + ) + ) end end diff --git a/app/mailers/convenor_contact_mailer.rb b/app/mailers/convenor_contact_mailer.rb index 5859399bda..3b1e71fd47 100644 --- a/app/mailers/convenor_contact_mailer.rb +++ b/app/mailers/convenor_contact_mailer.rb @@ -5,11 +5,12 @@ def request_project_membership(user, _convenor, unit, _first_name, _last_name) institution_email_domain = Doubtfire::Application.config.institution[:email_domain] admin_emails = User.admins.map(&:email) user_email = "#{user.username}@#{institution_email_domain}" - mail to: admin_emails, - from: user_email, - subject: "[#{@doubtfire_product_name}] Please add #{user.username} to #{unit.name}", - body: "The following user wishes to be added to #{unit.name} on " \ - "#{@doubtfire_product_name}:\n\nUsername: #{user.username}\nEmail: #{user_email}\n" \ - "Name: #{user.name}" + mail({ + to: admin_emails, + subject: "[#{@doubtfire_product_name}] Please add #{user.username} to #{unit.name}", + body: "The following user wishes to be added to #{unit.name} on " \ + "#{@doubtfire_product_name}:\n\nUsername: #{user.username}\nEmail: #{user_email}\n" \ + "Name: #{user.name}" + }.merge(outbound_sender_headers(development_from: user_email, reply_to: user_email))) end end diff --git a/app/mailers/d2l_result_mailer.rb b/app/mailers/d2l_result_mailer.rb index b50ff6bdb6..c066606a5a 100644 --- a/app/mailers/d2l_result_mailer.rb +++ b/app/mailers/d2l_result_mailer.rb @@ -15,6 +15,10 @@ def result_message(unit, user, result_message: 'completed', success: true) attachments['result.csv'] = File.read(path) end - mail(to: email, from: email, subject: "#{@doubtfire_product_name} #{unit.code} - D2L Grade Transfer Result") + mail( + { to: email, subject: "#{@doubtfire_product_name} #{unit.code} - D2L Grade Transfer Result" }.merge( + outbound_sender_headers(development_from: email) + ) + ) end end diff --git a/app/mailers/error_log_mailer.rb b/app/mailers/error_log_mailer.rb index 578a6f6fc3..e0e4226fc7 100644 --- a/app/mailers/error_log_mailer.rb +++ b/app/mailers/error_log_mailer.rb @@ -11,6 +11,10 @@ def error_message(subject, message, exception) backtrace = exception.backtrace&.join("\n") || 'No backtrace available' @error_log = "#{message}\n\n#{exception.message}\n\n#{backtrace}" - mail(to: email, from: email, subject: "#{@doubtfire_product_name} Error Log - #{subject}") + mail( + { to: email, subject: "#{@doubtfire_product_name} Error Log - #{subject}" }.merge( + outbound_sender_headers(development_from: email) + ) + ) end end diff --git a/app/mailers/notifications_mailer.rb b/app/mailers/notifications_mailer.rb index f4aefa1499..dcb813e3c6 100644 --- a/app/mailers/notifications_mailer.rb +++ b/app/mailers/notifications_mailer.rb @@ -1,10 +1,104 @@ class NotificationsMailer < ApplicationMailer + layout "notification_mail" + + # Project#top_tasks slices its result to five. The number is not exposed by the + # model, so it is named here rather than written into the copy of two templates + # where nobody would find it again. + TOP_TASK_LIMIT = 5 + def add_general @doubtfire_host = Doubtfire::Application.config.institution[:host] @doubtfire_product_name = Doubtfire::Application.config.institution[:product_name] @unsubscribe_url = "#{@doubtfire_host}/edit_profile" end + # Sends a single in-system notification as an email. Called by + # NotificationEmailJob, which lets delivery failures reach Sidekiq so they can + # be retried without blocking the request that created the notification. + def single_notification(notification) + add_general + + @notification = notification + @user = notification.user + @notification_url = notification_url_for(notification) + + # Use the deployment's SMTP-authorised sender, with a development-safe + # fallback for older installations that have not configured one yet. + from_address = Doubtfire::Application.config.institution[:email_sender].presence || 'noreply@doubtfire.local' + + email_with_name = address_with_name(@user) + subject = "#{@doubtfire_product_name}: #{SUBJECTS.fetch(notification.event, 'New notification')}" + + # An event may ship its own pair of templates named after it, for example + # task_comment_created.html.erb and task_comment_created.text.erb. Events + # without them fall back to the generic single_notification pair. + # + # This is why a new event ticket only ever adds files and never edits this + # method: eight event tickets can run in parallel without touching each + # other's work. + mail( + to: email_with_name, + subject: subject, + template_name: event_template_name(notification.event), + **outbound_sender_headers(development_from: from_address) + ).tap { |message| message.raise_delivery_errors = true } + end + + # Delivers a second, independent message to a verified additional address. + # It is intentionally not a CC: neither destination learns the other address, + # and a failure here can be isolated from the primary institutional delivery. + def additional_notification_copy(notification, address) + add_general + + @notification = notification + @user = notification.user + @notification_url = notification_url_for(notification) + + from_address = Doubtfire::Application.config.institution[:email_sender].presence || 'noreply@doubtfire.local' + subject = "#{@doubtfire_product_name}: #{SUBJECTS.fetch(notification.event, 'New notification')}" + + mail( + to: address, + subject: subject, + template_name: event_template_name(notification.event), + **outbound_sender_headers(development_from: from_address) + ).tap { |message| message.raise_delivery_errors = true } + end + SUBJECTS = { + 'task_comment_created' => 'New task comment', + 'task_status_changed' => 'Task status changed', + 'task_due_soon' => 'Task due soon', + 'task_due_date_changed' => 'Task due date changed', + 'new_task_available' => 'New task available', + 'task_submitted' => 'Task submitted', + 'extension_assessed' => 'Extension request assessed', + 'group_membership_changed' => 'Group membership changed', + 'discussion_request_created' => 'New discussion request', + 'portfolio_received' => 'Portfolio received', + 'tutorial_changed' => 'Tutorial changed', + 'portfolio_submitted' => 'Portfolio submitted', + 'task_automated_comment_created' => 'New note on a task', + 'unit_announcement_published' => 'New announcement', + 'unit_announcement_updated' => 'Announcement updated', + 'unit_session_changed' => 'Session changed', + 'unit_session_starting_soon' => 'Session starting soon' + }.freeze + + # The page the email's button opens: the same page the bell would open for + # this recipient, or nothing when the notification has nowhere to go. + def notification_url_for(notification) + path = notification.web_path + path.present? ? "#{@doubtfire_host}#{path}" : nil + end + + # The event's own template if it exists, otherwise the generic one. + def event_template_name(event) + return 'single_notification' if event.blank? + return 'single_notification' unless lookup_context.exists?(event, [self.class.mailer_name], false) + + event + end + def weekly_staff_summary(unit_role, summary_stats) return nil if unit_role.nil? @@ -37,11 +131,24 @@ def weekly_staff_summary(unit_role, summary_stats) @convenor = @unit.main_convenor_user @summary_stats = summary_stats - email_with_name = %("#{@staff.name}" <#{@staff.email}>) - convenor_email = %("#{@convenor.name}" <#{@convenor.email}>) - subject = "#{@unit.name}: Weekly Summary" + email_with_name = address_with_name(@staff) + convenor_email = address_with_name(@convenor) - mail(to: email_with_name, from: convenor_email, subject: subject) + # A distinct shape from the student subject, so the two never look like the + # same mail in the inbox of someone who is both. + waiting = @data[:tasks_awaiting_feedback_count].to_i + subject = + if waiting.positive? + "#{@unit.name} teaching: #{waiting} task#{'s' unless waiting == 1} waiting for your feedback" + else + "#{@unit.name} teaching: weekly summary" + end + + mail( + { to: email_with_name, subject: subject }.merge(bulk_list_headers).merge( + outbound_sender_headers(development_from: convenor_email, reply_to: convenor_email) + ) + ) end def weekly_student_summary(project, summary_stats, did_revert_to_pass) @@ -69,15 +176,93 @@ def weekly_student_summary(project, summary_stats, did_revert_to_pass) @sent_comments = project.comments.where("user_id = :student_id AND task_comments.created_at > :start", student_id: @student.id, start: Time.zone.now - 7.days).count @top_tasks = project.top_tasks - @overdue_top = @top_tasks.select { |tt| tt[:reason] == :overdue } - @soon_top = @top_tasks.select { |tt| tt[:reason] == :soon } - @ahead_top = @top_tasks.select { |tt| tt[:reason] == :ahead } + @top_tasks_truncated = @top_tasks.count >= TOP_TASK_LIMIT + + # What the student can actually act on. Two cheap reads and one grouped + # count, no query per task: the definitions the target grade asks for, the + # target date of each task the student already has, and how those tasks are + # spread across the statuses. top_tasks drops the date it sorted on, so the + # dates are looked up here and matched back by task definition. + assigned_defs = project.assigned_task_defs.select(:id, :target_date).to_a + @grade_task_total = assigned_defs.count + @task_due_dates = {} + status_by_definition = {} + project.tasks.each do |task| + @task_due_dates[task.task_definition_id] = task.due_date + status_by_definition[task.task_definition_id] = task.task_status_id + end + status_counts = project.assigned_tasks.group(:task_status_id).count + + # top_tasks orders by target grade band and then by where the definition sits + # in the unit, never by date, because that is the order the dashboard wants. + # An email that says "your oldest" has to sort by date itself, and the list, + # the lead and the subject all have to read the same order. + target_date_for = lambda do |entry| + @task_due_dates[entry[:task_definition].id] || entry[:task_definition].target_date + end + by_date = ->(entries) { entries.sort_by { |entry| target_date_for.call(entry) || Date.new(9999, 1, 1) } } + + @overdue_top = by_date.call(@top_tasks.select { |tt| tt[:reason] == :overdue }) + @soon_top = by_date.call(@top_tasks.select { |tt| tt[:reason] == :soon }) + @ahead_top = by_date.call(@top_tasks.select { |tt| tt[:reason] == :ahead }) + + # Ready for feedback is the one status that is not the student's move. Every + # other incomplete status is, which is the split the dashboard makes too. + @waiting_on_tutor = status_counts.fetch(TaskStatus.ready_for_feedback.id, 0) + @tasks_complete = status_counts.fetch(TaskStatus.complete.id, 0) + @waiting_on_student = [@grade_task_total - @tasks_complete - @waiting_on_tutor, 0].max + + # Work the tutor has already handed back. It is not the same as a task never + # opened, and it is the pile most worth clearing, so it gets counted apart. + @needs_your_response = weekly_returned_count(status_counts) + + # Every status the student's own tasks are in, in the order they matter, + # dropping the ones nobody is sitting on. A task definition with no task row + # has never been opened, so it joins the not started pile rather than + # vanishing from the total. + @status_summary = digest_status_summary(status_counts, @grade_task_total) + + # Pace, against the target schedule rather than against the hard deadline. + # The date on a task is its target date, adjusted for any extension, which is + # not the same thing as TaskDefinition#due_date, the date after which work + # stops being accepted. The copy has to say target wherever this is the + # number behind it. + # + # The comparison is strictly before today, matching top_tasks, so a task + # whose target date is today is not counted as having slipped. + passed, behind = weekly_target_pace(assigned_defs, status_by_definition) + @targets_passed = passed + + # Tasks past their target date and still not complete. This is the honest + # total, and it is not the same as the overdue list above: task_definitions_ + # and_status keeps only seven statuses, so a task sitting in time_exceeded, + # feedback_exceeded, attention_required, rediscuss or assess_in_portfolio is + # never eligible for that list however far past its date it is, and neither + # is one whose target grade is not among the unit's grade values. Without + # this number the email can say nothing is overdue and then draw a chart + # that disagrees. + @behind_target = behind + @on_target = passed - behind + + unit_end = project.unit.end_date + @weeks_left = unit_end.present? ? ((unit_end.to_date - Time.zone.today).to_f / 7).ceil : nil + @portfolio_due = project.unit.portfolio_auto_generation_date + @portfolio_days_left = @portfolio_due.present? ? (@portfolio_due.to_date - Time.zone.today).to_i : nil + + email_with_name = address_with_name(@student) + tutor_email = address_with_name(@tutor) - email_with_name = %("#{@student.name}" <#{@student.email}>) - tutor_email = %("#{@tutor.name}" <#{@tutor.email}>) - subject = "#{project.unit.name}: Weekly Summary" + # Every state used to send the same subject, so a student who filtered the + # quiet weeks lost the week they fell behind along with them. The subject now + # leads on whatever is worst, and only says "Weekly summary" when there is + # genuinely nothing outstanding. + subject = weekly_summary_subject(project.unit.name) - mail(to: email_with_name, from: tutor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge(bulk_list_headers).merge( + outbound_sender_headers(development_from: tutor_email, reply_to: tutor_email) + ) + ) end def top_task_desc(tt) @@ -108,8 +293,272 @@ def this_these(num) end end + # One email covering every unit a student is in, at the cadence they asked + # for. This is a sibling of weekly_student_summary, not a replacement: that one + # still sends per unit and still carries the unit's convenor in From and the + # unit's tutor in Reply-To, which a combined email cannot. + # + # The cadence is passed in rather than read off the user, so the scheduler owns + # who gets mailed and when, and this owns what the mail says. A run selects its + # recipients, then calls this once per student with the cadence it is running. + def student_digest(user, cadence = 'weekly') + return nil if user.nil? + + add_general + + @cadence = DIGEST_CADENCES.include?(cadence.to_s) ? cadence.to_s : 'weekly' + @window = DIGEST_WINDOWS.fetch(@cadence) + @window_start = @window[:length].ago + @student = user + + # What each cadence is for. Daily is a deadline list and nothing else, short + # enough to read on a lock screen. Weekly adds the standing picture. Monthly + # drops deadlines entirely: a date rendered as "in two days" is read up to a + # month after it was true, which teaches the reader that the email lies. + @show_deadlines = @cadence != 'monthly' + @show_standing = @cadence != 'daily' + @show_trend = @cadence == 'monthly' + + projects = Project.where(user: user, enrolled: true) + .joins(:unit).where(units: { active: true }) + .includes(:unit) + .to_a + @units = projects.filter_map { |project| digest_unit_summary(project) } + return nil if @units.empty? + + # A student's real question does not respect unit boundaries, so the thing + # they open the mail for is one list across all of them, in date order. + @next_up = @units + .flat_map { |summary| (summary[:overdue] + summary[:soon]).map { |entry| entry.merge(unit_summary: summary) } } + .sort_by { |entry| entry[:target_date] || Date.new(9999, 1, 1) } + .first(DIGEST_NEXT_UP_LIMIT) + + @overdue_total = @units.sum { |summary| summary[:behind_target] } + @needs_response_total = @units.sum { |summary| summary[:needs_your_response] } + @waiting_on_tutor_total = @units.sum { |summary| summary[:waiting_on_tutor] } + @tasks_complete_total = @units.sum { |summary| summary[:tasks_complete] } + @grade_task_total = @units.sum { |summary| summary[:grade_task_total] } + @completed_in_window = @units.sum { |summary| summary[:completed_in_window] } + @comments_in_window = @units.sum { |summary| summary[:comments_in_window] } + @remaining_total = [@grade_task_total - @tasks_complete_total, 0].max + + # The monthly projection. The soonest unit end date is the one that binds, so + # that is the horizon, and the rate is what they actually did this window. + soonest_end = @units.filter_map { |summary| summary[:unit].end_date }.min + @months_left = soonest_end.present? ? ((soonest_end.to_date - Time.zone.today).to_f / 30).round(1) : nil + if @show_trend && @months_left.present? && @months_left.positive? + @projected_finish = (@completed_in_window * @months_left).floor + @projected_shortfall = [@remaining_total - @projected_finish, 0].max + end + + # From and Reply-To. The per-unit mail could put a real person in both + # because it only ever spoke for one unit. This one spans several, each with + # its own convenor and its own tutor, and picking any of them would send a + # reply about one unit to the staff of another. So the institution sender + # carries it, no Reply-To is set, and every unit block names that unit's + # tutor with a mailto beside it. That is more correct than the single header + # ever was, not less. + from_address = Doubtfire::Application.config.institution[:email_sender].presence || 'noreply@doubtfire.local' + + mail( + { to: address_with_name(user), subject: digest_subject }.merge(bulk_list_headers).merge( + outbound_sender_headers(development_from: from_address) + ) + ) + end + helper_method :top_task_desc helper_method :were_was helper_method :are_is helper_method :this_these + + private + + DIGEST_CADENCES = %w[daily weekly monthly].freeze + DIGEST_NEXT_UP_LIMIT = 6 + + DIGEST_WINDOWS = { + 'daily' => { length: 1.day, noun: 'today', since: 'since yesterday' }, + 'weekly' => { length: 7.days, noun: 'this week', since: 'this week' }, + 'monthly' => { length: 30.days, noun: 'this month', since: 'over the last month' } + }.freeze + + # Everything one unit contributes to the digest. Same shape as the figures the + # per-unit weekly builds, so the two agree, but every window here comes from + # the cadence rather than from a hardcoded seven days. + def digest_unit_summary(project) + unit = project.unit + return nil if unit.nil? + + assigned_defs = project.assigned_task_defs.select(:id, :target_date).to_a + return nil if assigned_defs.empty? + + due_dates = {} + status_by_definition = {} + project.tasks.each do |task| + due_dates[task.task_definition_id] = task.due_date + status_by_definition[task.task_definition_id] = task.task_status_id + end + status_counts = project.assigned_tasks.group(:task_status_id).count + + complete_id = TaskStatus.complete.id + targets_passed = 0 + behind_target = 0 + assigned_defs.each do |definition| + target = due_dates[definition.id] || definition.target_date + next if target.blank? || target.to_date >= Time.zone.today + + targets_passed += 1 + behind_target += 1 unless status_by_definition[definition.id] == complete_id + end + + returned = [ + TaskStatus.fix_and_resubmit, TaskStatus.redo, TaskStatus.discuss, TaskStatus.rediscuss, + TaskStatus.demonstrate, TaskStatus.feedback_exceeded, TaskStatus.attention_required + ].sum { |status| status_counts.fetch(status.id, 0) } + + grade_task_total = assigned_defs.count + tasks_complete = status_counts.fetch(complete_id, 0) + waiting_on_tutor = status_counts.fetch(TaskStatus.ready_for_feedback.id, 0) + + entries = project.top_tasks.map do |entry| + definition = entry[:task_definition] + entry.merge( + project: project, + unit: unit, + target_date: due_dates[definition.id] || definition.target_date + ) + end + by_date = ->(list) { list.sort_by { |entry| entry[:target_date] || Date.new(9999, 1, 1) } } + + { + project: project, + unit: unit, + tutor: project.main_convenor_user, + overdue: by_date.call(entries.select { |entry| entry[:reason] == :overdue }), + soon: by_date.call(entries.select { |entry| entry[:reason] == :soon }), + ahead: by_date.call(entries.select { |entry| entry[:reason] == :ahead }), + grade_task_total: grade_task_total, + tasks_complete: tasks_complete, + waiting_on_tutor: waiting_on_tutor, + needs_your_response: returned, + waiting_on_student: [grade_task_total - tasks_complete - waiting_on_tutor, 0].max, + targets_passed: targets_passed, + behind_target: behind_target, + status_summary: digest_status_summary(status_counts, grade_task_total), + # Both of these used to be pinned to seven days whatever the caller wanted, + # which would have made them wrong rather than stale in a daily email. + completed_in_window: project.task_engagements + .where(engagement: TaskStatus.complete.name) + .where('task_engagements.engagement_time >= ?', @window_start) + .count, + comments_in_window: project.comments + .where('task_comments.user_id = :uid AND task_comments.created_at >= :start', + uid: project.user_id, start: @window_start) + .count + } + end + + # How many target dates have passed, and how many of those tasks are still not + # complete. Reads @task_due_dates, so it runs after weekly_student_summary fills it. + def weekly_target_pace(assigned_defs, status_by_definition) + complete_id = TaskStatus.complete.id + passed = 0 + behind = 0 + assigned_defs.each do |definition| + target = @task_due_dates[definition.id] || definition.target_date + next if target.blank? || target.to_date >= Time.zone.today + + passed += 1 + behind += 1 unless status_by_definition[definition.id] == complete_id + end + [passed, behind] + end + + def weekly_returned_count(status_counts) + returned_statuses = [ + TaskStatus.fix_and_resubmit, TaskStatus.redo, TaskStatus.discuss, + TaskStatus.rediscuss, TaskStatus.demonstrate, TaskStatus.feedback_exceeded, + TaskStatus.attention_required + ] + returned_statuses.sum { |status| status_counts.fetch(status.id, 0) } + end + + def weekly_summary_subject(unit_name) + if @behind_target.positive? + "#{unit_name}: #{@behind_target} task#{'s' unless @behind_target == 1} behind target" + elsif @needs_your_response.positive? + "#{unit_name}: #{@needs_your_response} task#{'s' unless @needs_your_response == 1} waiting on you" + elsif @soon_top.present? + "#{unit_name}: #{@soon_top.first[:task_definition].abbreviation} due this week" + else + "#{unit_name}: Weekly summary" + end + end + + # Shared by the per-unit weekly and the digest, so both list the same statuses. + def digest_status_summary(status_counts, grade_task_total) + ordered = [ + [TaskStatus.complete, 'complete'], + [TaskStatus.ready_for_feedback, 'with your tutor to mark'], + [TaskStatus.fix_and_resubmit, 'to fix and resubmit'], + [TaskStatus.redo, 'to redo'], + [TaskStatus.discuss, 'to talk through with your tutor'], + [TaskStatus.rediscuss, 'to talk through again'], + [TaskStatus.demonstrate, 'to demonstrate'], + [TaskStatus.feedback_exceeded, 'out of feedback attempts'], + [TaskStatus.attention_required, 'needing attention'], + [TaskStatus.time_exceeded, 'past the deadline'], + [TaskStatus.assess_in_portfolio, 'to carry into your portfolio'], + [TaskStatus.fail, 'marked fail'], + [TaskStatus.need_help, 'where you asked for help'], + [TaskStatus.working_on_it, 'you are working on'] + ] + summary = ordered.filter_map do |status, label| + count = status_counts.fetch(status.id, 0) + [status.status_key, label, count] if count.positive? + end + never_opened = grade_task_total - status_counts.values.sum + not_started = status_counts.fetch(TaskStatus.not_started.id, 0) + [never_opened, 0].max + summary << [:not_started, 'not opened yet', not_started] if not_started.positive? + summary + end + + # A subject that says which of the three this is and what state the student is + # in, so a filter on the quiet ones cannot swallow the week they fall behind. + def digest_subject + scope = "#{@doubtfire_product_name} #{@window[:noun]}" + + if @show_trend + return "#{scope}: #{@completed_in_window} task#{'s' unless @completed_in_window == 1} finished" if @completed_in_window.positive? + + return "#{scope}: nothing finished yet" + end + + if @overdue_total.positive? + "#{scope}: #{@overdue_total} task#{'s' unless @overdue_total == 1} behind target" + elsif @needs_response_total.positive? + "#{scope}: #{@needs_response_total} task#{'s' unless @needs_response_total == 1} waiting on you" + elsif @next_up.present? + "#{scope}: #{@next_up.first[:task_definition].abbreviation} next in #{@next_up.first[:unit].code}" + else + "#{scope}: nothing outstanding" + end + end + + # The weekly summaries go to every student and every staff member of every + # active unit on a schedule, which is bulk mail whatever it is about. Without + # List-Unsubscribe the only way out is a body link behind a login, and Gmail + # offers "report spam" where it would otherwise offer "unsubscribe". + # + # List-Unsubscribe-Post is deliberately not set. One-Click promises a URL that + # accepts an unauthenticated POST and acts on it, and the preferences page this + # points at is a logged-in page; advertising One-Click against it would have + # Gmail POST to something that cannot honour it, which is worse than not + # claiming it. It can be added the day a one-click endpoint exists. + def bulk_list_headers + return {} if @unsubscribe_url.blank? + + { 'List-Unsubscribe' => "<#{@unsubscribe_url}>" } + end end diff --git a/app/mailers/portfolio_evidence_mailer.rb b/app/mailers/portfolio_evidence_mailer.rb index 743503c25b..36382cf797 100644 --- a/app/mailers/portfolio_evidence_mailer.rb +++ b/app/mailers/portfolio_evidence_mailer.rb @@ -1,4 +1,6 @@ class PortfolioEvidenceMailer < ApplicationMailer + layout "notification_mail" + def add_general @doubtfire_host = Doubtfire::Application.config.institution[:host] @doubtfire_product_name = Doubtfire::Application.config.institution[:product_name] @@ -15,10 +17,14 @@ def task_pdf_failed(project, tasks) @tutor = project.main_convenor_user @convenor = project.main_convenor_user - email_with_name = %("#{@student.name}" <#{@student.email}>) - tutor_email = %("#{@tutor.name}" <#{@tutor.email}>) + email_with_name = address_with_name(@student) + tutor_email = address_with_name(@tutor) subject = "#{project.unit.code} #{project.unit.name}: Task submission processing failed" - mail(to: email_with_name, from: tutor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: tutor_email, reply_to: tutor_email) + ) + ) end def task_pdf_ready_message(project, tasks) @@ -31,10 +37,14 @@ def task_pdf_ready_message(project, tasks) @tutor = project.main_convenor_user @convenor = project.main_convenor_user - email_with_name = %("#{@student.name}" <#{@student.email}>) - tutor_email = %("#{@tutor.name}" <#{@tutor.email}>) + email_with_name = address_with_name(@student) + tutor_email = address_with_name(@tutor) subject = "#{project.unit.name}: Task PDFs ready to view" - mail(to: email_with_name, from: tutor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: tutor_email, reply_to: tutor_email) + ) + ) end def task_feedback_ready(project, tasks) @@ -48,10 +58,14 @@ def task_feedback_ready(project, tasks) @has_comments = !@tasks.select { |t| t.is_last_comment_by?(@tutor) }.empty? return nil if @tutor.nil? || @student.nil? - email_with_name = %("#{@student.name}" <#{@student.email}>) - tutor_email = %("#{@tutor.name}" <#{@tutor.email}>) + email_with_name = address_with_name(@student) + tutor_email = address_with_name(@tutor) subject = "#{project.unit.name}: Feedback ready to review" - mail(to: email_with_name, from: tutor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: tutor_email, reply_to: tutor_email) + ) + ) end def overseer_assessment_failed(project, tasks) @@ -64,10 +78,14 @@ def overseer_assessment_failed(project, tasks) @tutor = project.main_convenor_user return nil if @tutor.nil? || @student.nil? - email_with_name = %("#{@student.name}" <#{@student.email}>) - tutor_email = %("#{@tutor.name}" <#{@tutor.email}>) + email_with_name = address_with_name(@student) + tutor_email = address_with_name(@tutor) subject = "#{project.unit.code} #{project.unit.name}: Automated feedback needs your attention" - mail(to: email_with_name, from: tutor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: tutor_email, reply_to: tutor_email) + ) + ) end def portfolio_ready(project) @@ -79,10 +97,14 @@ def portfolio_ready(project) @project = project @convenor = project.main_convenor_user - email_with_name = %("#{@student.name}" <#{@student.email}>) - convenor_email = %("#{@convenor.name}" <#{@convenor.email}>) + email_with_name = address_with_name(@student) + convenor_email = address_with_name(@convenor) subject = "#{project.unit.name}: Portfolio ready to review" - mail(to: email_with_name, from: convenor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: convenor_email, reply_to: convenor_email) + ) + ) end def portfolio_failed(project) @@ -94,9 +116,13 @@ def portfolio_failed(project) @project = project @convenor = project.main_convenor_user - email_with_name = %("#{@student.name}" <#{@student.email}>) - convenor_email = %("#{@convenor.name}" <#{@convenor.email}>) + email_with_name = address_with_name(@student) + convenor_email = address_with_name(@convenor) subject = "#{project.unit.name}: Portfolio failed to compile" - mail(to: email_with_name, from: convenor_email, subject: subject) + mail( + { to: email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: convenor_email, reply_to: convenor_email) + ) + ) end end diff --git a/app/mailers/tutor_note_mailer.rb b/app/mailers/tutor_note_mailer.rb index 094644aa79..b097a0f0e9 100644 --- a/app/mailers/tutor_note_mailer.rb +++ b/app/mailers/tutor_note_mailer.rb @@ -1,4 +1,6 @@ class TutorNoteMailer < ApplicationMailer + layout "notification_mail" + def add_general @doubtfire_host = Doubtfire::Application.config.institution[:host] @doubtfire_product_name = Doubtfire::Application.config.institution[:product_name] @@ -18,10 +20,14 @@ def notify_tutor_note(tutor_note, recipient) @task = tutor_note.task - recipient_email_with_name = %("#{recipient.name}" <#{recipient.email}>) - tutor_email = %("#{@from.name}" <#{@from.email}>) + recipient_email_with_name = address_with_name(recipient) + tutor_email = address_with_name(@from) subject = "#{@unit.name}: New tutor note from #{@from.name}" - mail(to: recipient_email_with_name, from: tutor_email, subject: subject) + mail( + { to: recipient_email_with_name, subject: subject }.merge( + outbound_sender_headers(development_from: tutor_email, reply_to: tutor_email) + ) + ) end end diff --git a/app/models/additional_notification_email.rb b/app/models/additional_notification_email.rb new file mode 100644 index 0000000000..b90e29df6d --- /dev/null +++ b/app/models/additional_notification_email.rb @@ -0,0 +1,79 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmail < ApplicationRecord + TOKEN_PURPOSE = 'additional-notification-email-verification' + EMAIL_FORMAT = /\A([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})\z/i + + belongs_to :user + + before_validation :normalise_email + + validates :email, + presence: true, + length: { maximum: 254 }, + format: { with: EMAIL_FORMAT } + validates :verification_version, + numericality: { only_integer: true, greater_than_or_equal_to: 1 } + validate :must_differ_from_primary_email + + scope :verified, -> { where.not(verified_at: nil) } + + def pending? + verified_at.nil? + end + + def verified? + verified_at.present? + end + + def verification_expired? + verification_expires_at.blank? || verification_expires_at <= Time.current + end + + # The token is signed rather than stored. Jobs carry only this record's id + # and version, while a replacement or resend increments the version and + # immediately invalidates every earlier link. + def verification_token + raise 'Cannot issue an expired verification token' if verification_expired? + + self.class.token_verifier.generate( + { id: id, version: verification_version }, + expires_in: verification_expires_at - Time.current, + purpose: TOKEN_PURPOSE + ) + end + + def self.record_for_token(token) + payload = token_verifier.verified(token.to_s, purpose: TOKEN_PURPOSE) + return nil unless payload.is_a?(Hash) + + record = find_by(id: payload['id'] || payload[:id]) + return nil if record.nil? || record.verification_expired? + return nil unless record.verification_version == (payload['version'] || payload[:version]).to_i + + record + rescue ActiveSupport::MessageVerifier::InvalidSignature + nil + end + + def self.token_verifier + ActiveSupport::MessageVerifier.new( + Rails.application.secret_key_base, + digest: 'SHA256', + serializer: JSON + ) + end + + private + + def normalise_email + self.email = email.to_s.strip.downcase + end + + def must_differ_from_primary_email + return if user.nil? || email.blank? + return unless email.casecmp?(user.email.to_s.strip) + + errors.add(:email, 'must be different from the institutional email') + end +end diff --git a/app/models/additional_notification_email_audit.rb b/app/models/additional_notification_email_audit.rb new file mode 100644 index 0000000000..59d7236aef --- /dev/null +++ b/app/models/additional_notification_email_audit.rb @@ -0,0 +1,18 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailAudit < ApplicationRecord + EVENTS = %w[ + verification_requested + address_replaced + verification_resent + verification_email_delivered + verified + removed + notification_copy_delivered + notification_copy_failed + ].freeze + + belongs_to :user + + validates :event, presence: true, inclusion: { in: EVENTS } +end diff --git a/app/models/application_record.rb b/app/models/application_record.rb index 88384bd4d3..190ee85daa 100644 --- a/app/models/application_record.rb +++ b/app/models/application_record.rb @@ -1,4 +1,45 @@ class ApplicationRecord < ActiveRecord::Base include LogHelper self.abstract_class = true + + def self.current_read_prefix + connection = self.connection + @current_read_prefixes ||= {} + key = [connection.adapter_name, connection.database_version.to_s] + unless @current_read_prefixes.key?(key) + supported = connection.respond_to?(:mariadb?) && connection.mariadb? && + connection.select_rows("SHOW VARIABLES LIKE 'innodb_snapshot_isolation'").any? + @current_read_prefixes[key] = supported ? 'SET STATEMENT innodb_snapshot_isolation=OFF FOR ' : '' + end + @current_read_prefixes.fetch(key) + end + private_class_method :current_read_prefix + + def self.current_rows(relation) + # MariaDB snapshot isolation rejects a locking read of newer committed rows. + # Delivery admission explicitly needs current rows for its locks. The statement + # override leaves the surrounding transaction/session configuration intact; + # older MariaDB and MySQL use their ordinary locking-read behavior. + sql = "#{current_read_prefix}#{relation.lock.to_sql}" + uncached { find_by_sql(sql) } + end + + def self.with_current_row_access + return yield if current_read_prefix.empty? + + connection = self.connection + previous = connection.select_value('SELECT @@SESSION.innodb_snapshot_isolation').to_i + return yield if previous.zero? + + # Foreign-key checks during INSERT also acquire locks. Limit this MariaDB + # compatibility setting to notification reservation and always restore it + # before the caller's surrounding workflow resumes. Locks and foreign-key + # constraints remain enabled; ordinary MySQL needs no override. + begin + connection.execute('SET SESSION innodb_snapshot_isolation = OFF') + yield + ensure + connection.execute('SET SESSION innodb_snapshot_isolation = ON') + end + end end diff --git a/app/models/comments/extension_comment.rb b/app/models/comments/extension_comment.rb index abd9d1030c..33f1e9c9a2 100644 --- a/app/models/comments/extension_comment.rb +++ b/app/models/comments/extension_comment.rb @@ -1,6 +1,21 @@ class ExtensionComment < TaskComment belongs_to :assessor, class_name: 'User', optional: true + # The status that triggered a resubmission extension. It is nil on extensions + # a student asked for, which is what tells the two kinds apart. + belongs_to :task_status, optional: true + + # An extension OnTrack worked out for itself when staff sent the task back for + # more work, rather than one a student asked for. + # + # Do not call this "automatic". #assess_extension already uses that word for + # something else, an extension a student requested that was approved without a + # person weighing it up, and one word meaning two things in one class is how + # the wrong branch gets taken. + def resubmission_extension? + task_status.present? + end + def serialize(user) json = super(user) json[:granted] = extension_granted @@ -9,6 +24,8 @@ def serialize(user) json[:weeks_requested] = extension_weeks json[:extension_response] = extension_response json[:task_status] = task.status + json[:resubmission_extension] = resubmission_extension? + json[:source_status] = resubmission_extension? ? task_status.status_key : nil json end @@ -27,26 +44,40 @@ def mark_as_read(user, unit = self.unit) super if assessed? || user == project.student || user != recipient end - def assess_extension(user, granted, automatic = false) + # Assess an extension a student asked for. `auto_approved` says the unit + # approved it without a person weighing it up, which only changes the wording + # the student sees. It is not the same idea as #resubmission_extension?, which + # is about where the extension came from rather than who signed it off. + def assess_extension(user, granted, auto_approved = false) if self.assessed? - self.errors[:extension] << 'has already been assessed' + errors.add(:extension, 'could not be applied') + return false + end + + can_apply = self.task.can_apply_for_extension? + should_grant = granted && can_apply + + if should_grant && !self.task.grant_extension(user, extension_weeks) + errors.add(:extension, 'could not be applied') return false end self.assessor = user self.date_extension_assessed = Time.zone.now - self.extension_granted = granted && self.task.can_apply_for_extension? + self.extension_granted = should_grant + + should_notify = true if self.extension_granted - self.task.grant_extension(user, extension_weeks) - if automatic + if auto_approved self.extension_response = "Time extended to #{self.task.due_date.strftime('%a %b %e')}" else self.extension_response = "Extension granted to #{self.task.due_date.strftime('%a %b %e')}" end - elsif !self.task.can_apply_for_extension? && granted + elsif !can_apply && granted self.extension_response = "Extension cannot be granted as deadline has been reached" - errors[:extension] << 'cannot be granted as deadline has been reached' + errors.add(:extension, 'cannot be granted as deadline has been reached') + should_notify = false else self.extension_response = "Extension rejected" end @@ -54,5 +85,25 @@ def assess_extension(user, granted, automatic = false) # Now make sure to read it by the main tutor - even if assessed by someone else super_mark_as_read(project.tutor_for(task.task_definition)) save! + + if should_notify + begin + NotificationService.notify( + user: project.student, + type: 'extension', + event: 'extension_assessed', + message: extension_response, + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task.task_definition.abbreviation)}", + # The assessed request itself. An extension comment is a task comment, + # so reading the task's discussion clears this the same way it clears + # any other comment notification. + notifiable: self + ) + rescue StandardError => e + Rails.logger.error "Failed to notify student about extension assessment: #{e.message}" + end + end + + true end end diff --git a/app/models/group.rb b/app/models/group.rb index fec42947a6..dfab68744b 100644 --- a/app/models/group.rb +++ b/app/models/group.rb @@ -135,24 +135,27 @@ def switch_to_tutorial tutorial if group_set.keep_groups_in_same_class && has_active_group_members? projects.each do |proj| - # We need to remove members to break the circular dependency and switch tutorial - remove_member(proj) + # These membership changes are temporary while moving tutorial, + # so they must not generate leave/join notifications. + remove_member(proj, notify: false) te = proj.enrol_in tutorial unless te.valid? raise "Unable to move group as #{proj.student.name} could not switch tutorial." end - add_member(proj) + add_member(proj, notify: false) end end self.save! end end - def add_member(project) + def add_member(project, notify: true) gm = project.group_membership_for_groupset(group_set) + membership_changed = gm.nil? || !gm.active? || gm.group_id != id + if gm.nil? gm = GroupMembership.create(group: self, project: project) group_memberships << gm @@ -164,16 +167,42 @@ def add_member(project) gm.active = true gm.save! + notify_group_membership_change(project, 'added to') if notify && membership_changed + gm end - def remove_member(project) + def remove_member(project, notify: true) gm = group_memberships.where(project: project).first + was_active = gm.active? + gm.active = false - gm.save + saved = gm.save + + notify_group_membership_change(project, 'removed from') if notify && saved && was_active + self end + def notify_group_membership_change(project, change) + student = project.student + return if student.blank? + + NotificationService.notify( + user: student, + type: 'general', + event: 'group_membership_changed', + message: "You have been #{change} group #{name} in #{unit.code}.", + link: "/projects/#{project.id}/groups", + notifiable: self + ) + rescue StandardError => e + logger.error( + "Failed to raise group_membership_changed notification for project #{project.id}: #{e.message}" + ) + end + + private :notify_group_membership_change # # check if the project is the same as the current submission # diff --git a/app/models/notification.rb b/app/models/notification.rb new file mode 100644 index 0000000000..85dc3a9ae1 --- /dev/null +++ b/app/models/notification.rb @@ -0,0 +1,191 @@ +class Notification < ApplicationRecord + belongs_to :user + + # Only set on a unique-event conflict. Its push retry must leave the caller's + # older transaction snapshot before reading/updating the winning row again. + attr_accessor :reserved_from_existing_event + + # What the notification is about: the comment, the task, or whatever record + # the event happened to. Optional, because a `general` notification points at + # nothing and because every row raised before this column existed has no + # target. Polymorphic, because portfolio and extension events do not point at + # a task. + belongs_to :notifiable, polymorphic: true, optional: true + + # Notification categories. task, feedback, portfolio and unit_hub map onto the + # user preference columns in PREFERENCE_FOR_TYPE, so one category toggle gates + # every delivery channel (in-app, email, push). unit_hub also has email and + # push opt-ins of its own, see CHANNEL_PREFERENCES_FOR_TYPE. + TYPES = %w[task feedback portfolio extension general unit_hub].freeze + + # `notification_type` is the category the user's preferences switch on. + # `event` is the specific thing that happened within that category, e.g. + # 'task_comment_created'. It is free text so a new event ticket does not have + # to edit this model, but it is required so every notification can be traced + # back to the code that raised it. + + # Maps a notification type to the user preference column that gates it. + # Types without an entry here are always delivered. + PREFERENCE_FOR_TYPE = { + 'task' => :receive_task_notifications, + 'feedback' => :receive_feedback_notifications, + 'portfolio' => :receive_portfolio_notifications, + 'unit_hub' => :receive_unit_hub_notifications + }.freeze + + # Categories whose email and push channels are separate opt-ins. The column in + # PREFERENCE_FOR_TYPE still switches the whole category off, so these only + # narrow a category that is on. A type without an entry sends on every + # channel, which is how the older categories have always behaved. + CHANNEL_PREFERENCES_FOR_TYPE = { + 'unit_hub' => { + email: :receive_unit_hub_email_notifications, + push: :receive_unit_hub_push_notifications + } + }.freeze + + validates :notification_type, presence: true, inclusion: { in: TYPES } + validates :event, presence: true, length: { maximum: 255 } + validates :message, presence: true, length: { maximum: 500 } + validates :dedupe_key, length: { maximum: 191 }, allow_nil: true + + # Queue the email only once the transaction that created the notification has + # committed. Several callers raise notifications from inside a transaction, + # for example a tutorial enrolment being destroyed removes the student from + # their group, and a worker that picked the job up before the commit could not + # see the row yet. + after_commit :queue_email_delivery, on: :create + + before_create do + self.email_delivery_state = 'pending' if email_delivery_state == 'untracked' + end + + scope :unread, -> { where(read_at: nil) } + scope :recent_first, -> { order(created_at: :desc) } + + def read? + read_at.present? + end + + def mark_read! + update!(read_at: Time.zone.now) unless read? + end + + PROJECT_LINK = %r{\A/projects/(\d+)(?:/|\z)} + TASK_LINK = %r{\A/projects/\d+/dashboard/([^/]+)} + + TARGET_KEYS = %i[ + unit_id project_id student_id task_definition_id task_definition_abbr task_id comment_id group_id + announcement_id session_id + ].freeze + + # The ids a client needs to open the page this notification is about. + # + # Worked out when the notification is read, not stored, so a notification + # raised before these fields existed still gets them, and a record deleted + # since then comes back as nil rather than as an id that points at nothing. + # The notifiable is used first, and the link fills in whatever it cannot + # say, for example the project behind a group change or a new task. + def target_ids + @target_ids ||= resolve_target_ids + end + + STUDENT_FEEDBACK_EVENTS = %w[ + task_comment_created + task_automated_comment_created + discussion_request_created + extension_assessed + ].freeze + PORTFOLIO_EVENTS = %w[portfolio_received portfolio_submitted].freeze + + # The in-app page this notification opens, for the person it was sent to. + # + # The same decision the web client makes in notification-target.ts, kept here + # so an email button lands on the page the bell would have opened. A student + # goes to their own project, staff go to the task inbox or the staff + # portfolio view. Falls back to link when the ids cannot be worked out, which + # is also what an email for a deleted record gets. + def web_path + ids = target_ids + return link if ids[:project_id].nil? + + abbreviation = ids[:task_definition_abbr] + return link if abbreviation.nil? && link.to_s.match?(TASK_LINK) + + ids[:student_id] == user_id ? student_web_path(ids, abbreviation) : staff_web_path(ids, abbreviation) + end + + private + + def student_web_path(ids, abbreviation) + project = "/projects/#{ids[:project_id]}" + return "#{project}/groups" if event == 'group_membership_changed' + return "#{project}/tutorials" if event == 'tutorial_changed' + return "#{project}/portfolio" if PORTFOLIO_EVENTS.include?(event) || (notification_type == 'portfolio' && abbreviation.nil?) + return "#{project}/dashboard" if abbreviation.nil? + + task = "#{project}/dashboard/#{ERB::Util.url_encode(abbreviation)}" + STUDENT_FEEDBACK_EVENTS.include?(event) ? "#{task}/feedback" : task + end + + def staff_web_path(ids, abbreviation) + unit = "/units/#{ids[:unit_id]}" + if PORTFOLIO_EVENTS.include?(event) || (notification_type == 'portfolio' && abbreviation.nil?) + return "#{unit}/students/portfolios/#{ids[:project_id]}" + end + return "#{unit}/students" if abbreviation.nil? + + "#{unit}/tasks/inbox/#{ids[:student_id]}/#{ERB::Util.url_encode(abbreviation)}?students=all" + end + + def resolve_target_ids + ids = TARGET_KEYS.index_with { nil } + return unit_hub_target_ids(ids) if notifiable_type.in?(UNIT_HUB_NOTIFIABLES) + + comment = notifiable if notifiable.is_a?(TaskComment) + task = comment ? comment.task : (notifiable if notifiable.is_a?(Task)) + group = notifiable if notifiable.is_a?(Group) + project = notifiable if notifiable.is_a?(Project) + project ||= task&.project + project ||= Project.find_by(id: Regexp.last_match(1)) if link.to_s =~ PROJECT_LINK + + ids[:comment_id] = comment&.id + ids[:group_id] = group&.id + return ids if project.nil? + + ids[:project_id] = project.id + ids[:unit_id] = project.unit_id + ids[:student_id] = project.user_id + + task_definition = task&.task_definition + if task_definition.nil? && link.to_s =~ TASK_LINK + abbreviation = CGI.unescape(Regexp.last_match(1)) + task_definition = project.unit.task_definitions.find_by(abbreviation: abbreviation) + end + return ids if task_definition.nil? + + ids[:task_definition_id] = task_definition.id + ids[:task_definition_abbr] = task_definition.abbreviation + ids[:task_id] = task&.id || project.tasks.where(task_definition_id: task_definition.id).pick(:id) + ids + end + + UNIT_HUB_NOTIFIABLES = %w[UnitAnnouncement UnitLearningSession].freeze + + # A Unit Hub notification opens the hub for its unit, so it only needs the + # unit and the announcement or session. A deleted record leaves all of them + # nil, which the web client reads as no longer available. + def unit_hub_target_ids(ids) + record = notifiable + return ids if record.nil? + + ids[:unit_id] = record.unit_id + key = record.is_a?(UnitAnnouncement) ? :announcement_id : :session_id + ids[key] = record.id + ids + end + + def queue_email_delivery + NotificationService.queue_email(self) + end +end diff --git a/app/models/push_subscription.rb b/app/models/push_subscription.rb new file mode 100644 index 0000000000..63daeaef98 --- /dev/null +++ b/app/models/push_subscription.rb @@ -0,0 +1,139 @@ +# One browser registered to receive web push notifications. +# +# The endpoint is the URL the push service gave that browser. It identifies the +# browser, not the person, so it is unique across the whole table: if the same +# browser signs in as a different user the registration moves across instead of +# being duplicated. PushSubscriptionsApi does that move. +# +# The endpoint arrives from the client and the api later makes an outbound POST +# to it, so it is not free text. It has to be an https URL belonging to a push +# service we recognise, or a signed in user could point the api at an internal +# host and use it to make requests on their behalf. See PUSH_SERVICE_HOSTS. +class PushSubscription < ApplicationRecord + # Exact hosts. One per push service. + # + # fcm.googleapis.com Chrome, Opera, Brave + # android.googleapis.com older Chrome on Android + # updates.push.services.mozilla.com Firefox + # + # A verified Edge 151 subscription on macOS used WNS even though Edge is + # Chromium. Endpoint selection can vary by platform or release, so these + # labels are observations rather than a browser-detection contract. + PUSH_SERVICE_HOSTS = %w[ + fcm.googleapis.com + android.googleapis.com + updates.push.services.mozilla.com + ].freeze + + # Suffixes, for the services that shard across per-region subdomains. Matched + # with a leading dot so "evil-notify.windows.com" cannot pass as a subdomain + # of "notify.windows.com". + # + # *.notify.windows.com WNS, current Edge observed + # *.push.services.microsoft.com WNS, current + # *.push.apple.com Safari, iOS 16.4+ + # + # Not legacy. Edge 151 on macOS subscribed through + # wns2-bl2p.notify.windows.com when this was checked on 27 Aug 2026. Do not + # infer a browser only from an endpoint host; the testing guide records the + # scoped observation and the allow-list accepts the supported services. + PUSH_SERVICE_HOST_SUFFIXES = %w[ + .notify.windows.com + .push.services.microsoft.com + .push.apple.com + ].freeze + + belongs_to :user + + validates :endpoint, presence: true, uniqueness: true, length: { maximum: 500 } + validates :p256dh, presence: true, length: { maximum: 255 } + validates :auth, presence: true, length: { maximum: 255 } + + # A p256dh is a 65-byte uncompressed prime256v1 public key and an auth is a + # 16-byte secret, both base64url from the browser's PushSubscription. The + # length validations above only bound the string; a key of the wrong charset + # or decoded length still passes them and then fails deep inside web-push's + # encryption with an error that is neither ExpiredSubscription nor + # InvalidSubscription. Nothing retires such a row, so every push to that user + # fails the fan-out and Sidekiq retries, re-hitting the healthy devices too. + # Check the decoded shape on the way in. + P256DH_BYTES = 65 + AUTH_BYTES = 16 + + validate :endpoint_is_a_known_push_service + validate :encryption_keys_are_usable + + # Accept base64url material of the expected size, and validate the actual + # curve point for a public key. A correctly sized byte string alone can + # still fail encryption and prevent delivery to the user's other devices. + # + # Also called at delivery time, because rows written before this validation + # existed were never checked. Keep it a class method for that reason, the same + # as push_service_endpoint?. + def self.valid_web_push_key?(value, expected_bytes) + return false unless value.is_a?(String) && value.present? + + normalized = value.tr('-_', '+/') + normalized = normalized.ljust((normalized.length + 3) & ~3, '=') + decoded = Base64.strict_decode64(normalized) + return false unless decoded.bytesize == expected_bytes + return true unless expected_bytes == P256DH_BYTES + return false unless decoded.getbyte(0) == 4 + + group = OpenSSL::PKey::EC::Group.new('prime256v1') + point = OpenSSL::PKey::EC::Point.new(group, OpenSSL::BN.new(decoded, 2)) + point.on_curve? && !point.infinity? + rescue ArgumentError, OpenSSL::OpenSSLError + false + end + + # True when this endpoint is one we are willing to send to. + # + # Also called at delivery time, because rows written before this validation + # existed were never checked. Keep it a class method for that reason. + def self.push_service_endpoint?(endpoint) + return false if endpoint.blank? + + uri = URI.parse(endpoint.to_s) + + # https only. http would send the encrypted payload in the clear and is not + # something any real push service offers. + return false unless uri.is_a?(URI::HTTPS) + + # user:password@host is a redirect trick, and a non standard port is a sign + # somebody is aiming this somewhere it should not go. No push service uses + # either. + return false if uri.userinfo.present? + return false unless uri.port == 443 + + host = uri.host.to_s.downcase + return false if host.blank? + + PUSH_SERVICE_HOSTS.include?(host) || + PUSH_SERVICE_HOST_SUFFIXES.any? { |suffix| host.end_with?(suffix) } + rescue URI::InvalidURIError + false + end + + private + + def endpoint_is_a_known_push_service + return if endpoint.blank? # presence validation already covers this + + return if self.class.push_service_endpoint?(endpoint) + + errors.add(:endpoint, 'must be an https URL belonging to a recognised push service') + end + + def encryption_keys_are_usable + # Blank is left to the presence validations so a missing key is not reported + # twice. + if p256dh.present? && !self.class.valid_web_push_key?(p256dh, P256DH_BYTES) + errors.add(:p256dh, 'must be a base64url prime256v1 public key') + end + + if auth.present? && !self.class.valid_web_push_key?(auth, AUTH_BYTES) + errors.add(:auth, 'must be a 16-byte base64url secret') + end + end +end diff --git a/app/models/task.rb b/app/models/task.rb index a48148c8a3..86758e4d8b 100644 --- a/app/models/task.rb +++ b/app/models/task.rb @@ -6,6 +6,9 @@ class Task < ApplicationRecord include ApplicationHelper include GradeHelper + SUBMISSION_PROCESSING_STATES = %w[queued processing ready failed].freeze + SUBMISSION_PROCESSING_TIMEOUT = 10.minutes + # # Permissions around task data # @@ -129,6 +132,11 @@ def specific_permission_hash(role, perm_hash, _other) has_one :overflow_task_claim, dependent: :destroy has_many :comments, class_name: 'TaskComment', dependent: :destroy, inverse_of: :task + # The general comments association above owns deletion for this read-only subset. + has_many :granted_extension_comments, lambda { + where(extension_granted: true).where.not(date_extension_assessed: nil) + .order(date_extension_assessed: :desc, id: :desc) + }, class_name: 'ExtensionComment', inverse_of: :task, dependent: nil has_many :task_similarities, class_name: 'TaskSimilarity', dependent: :destroy, inverse_of: :task has_many :reverse_jplag_similarities, class_name: 'JplagTaskSimilarity', dependent: :destroy, inverse_of: :other_task, foreign_key: 'other_task_id' has_many :reverse_moss_similarities, class_name: 'MossTaskSimilarity', dependent: :destroy, inverse_of: :other_task, foreign_key: 'other_task_id' @@ -140,6 +148,11 @@ def specific_permission_hash(role, perm_hash, _other) has_many :test_attempts, dependent: :destroy has_many :session_activities, dependent: :destroy + # Notifications that point at this task. They go with it, because a + # notification about a record that no longer exists can never be reached by + # the reader and so can never be cleared by reading it. + has_many :notifications, as: :notifiable, dependent: :destroy, inverse_of: :notifiable + delegate :unit, to: :project delegate :student, to: :project delegate :upload_requirements, to: :task_definition @@ -221,10 +234,37 @@ def all_comments end end + # Reading the comments is also reading the notifications that pointed at them. + # + # The rule lives here rather than in the comments endpoint because this method + # is what "the user has now seen these comments" means, and it has more than + # one caller. A rule in one endpoint is a rule the next caller forgets. + # + # Only this user's notifications are touched, so a tutor opening a student's + # task cannot clear the student's bell. Only unread ones are touched, so a + # notification read three days ago keeps its original timestamp. One + # update_all over the whole set, because the loop above is already one write + # per comment. + # + # The type/id pairs are built by Rails from the records themselves rather than + # written out as SQL strings, so an STI subclass or a renamed class cannot + # leave a literal that still compiles and quietly matches nothing. def mark_comments_as_read(user, comments) + comments = comments.to_a + comments.each do |comment| comment.mark_as_read(user, unit) end + + unread = user.notifications.unread + # rubocop:disable Rails/SkipsModelValidations + # update_all on purpose. Setting read_at has no validation or callback to + # run, and the loop above is already one write per comment, so this is one + # statement rather than another N. + unread.where(notifiable: comments) + .or(unread.where(notifiable: self)) + .update_all(read_at: Time.zone.now) + # rubocop:enable Rails/SkipsModelValidations end def mark_comments_as_unread(user, comments) @@ -245,6 +285,10 @@ def comments_for_user(user) 'task_comments.id AS id', 'task_comments.comment AS comment', 'task_comments.content_type AS content_type', + 'task_comments.attachment_extension AS attachment_extension', + 'task_comments.attachment_original_filename AS attachment_original_filename', + 'task_comments.attachment_content_type AS attachment_content_type', + 'task_comments.attachment_byte_size AS attachment_byte_size', "case when u_crr.created_at IS NULL then 1 else 0 end AS is_new", 'r_crr.created_at AS recipient_read_time', 'task_comments.created_at AS created_at', @@ -292,13 +336,397 @@ def processing_pdf? folder_exists_in_new? || folder_exists_in_process? end + def submission_pdf_ready? + path = final_pdf_path + return false if path.blank? || !File.file?(path) + + # A previous attempt's PDF can remain in place while a replacement is + # queued. Do not advertise that stale artifact as the current submission. + # The timestamp comparison also lets a worker from the previous release + # prove that it completed even though it does not know about the state + # columns introduced by this release. Conversion clears the working + # folders before ready is recorded, so staged files alongside ready belong + # to a newer attempt that a previous-release process accepted. + return !processing_pdf? if submission_processing_state == 'ready' + + started_at = submission_processing_started_at + started_at.blank? || File.mtime(path) >= started_at + rescue SystemCallError + false + end + + def submission_files_ready? + has_done_file? || uncompressed_done_files? + end + + # Older submissions can keep their done files as a folder instead of a zip. + def uncompressed_done_files? + # A group task has no done folder until its group has submitted, and the + # path cannot be resolved without a group submission. + return false if group_task? && group_submission.nil? + + done_dir = student_work_dir(:done, false) + done_dir.present? && Dir.exist?(done_dir) && Dir.children(done_dir).any? + rescue SystemCallError + false + end + + def submission_pdf_replaced_after_failure? + finished_at = submission_processing_finished_at + path = final_pdf_path + finished_at.present? && path.present? && File.file?(path) && File.mtime(path) > finished_at + rescue SystemCallError + false + end + + def effective_submission_processing_state(now: Time.current) + state = submission_processing_state.presence + started_at = submission_processing_started_at + + # Rolling-deploy fallback for accepted work created before the state columns + # were populated. The filesystem remains authoritative for actual artefacts. + state = 'processing' if state.blank? && folder_exists_in_process? + state = 'queued' if state.blank? && folder_exists_in_new? + state = 'ready' if state.blank? && submission_pdf_ready? && !processing_pdf? + state = 'failed' if state.blank? && submission_date.present? && submission_files_ready? + state ||= 'not_submitted' + + # A PDF written after the failure was recorded, for example by the batch + # feedback import, means the submission has a usable PDF again. + state = 'ready' if state == 'failed' && !processing_pdf? && submission_pdf_replaced_after_failure? + + # During a rolling deploy an old worker can successfully create the new PDF + # without updating the durable state. Only accept it when the artifact is + # from this attempt and the working directory has been cleared. + if %w[queued processing].include?(state) && submission_pdf_ready? && !processing_pdf? + state = 'ready' + elsif state == 'ready' && processing_pdf? + # Likewise a previous-release API can stage a newer upload without + # touching the state columns. Report that attempt, timed from its files + # rather than from the attempt that reached ready. + state = folder_exists_in_process? ? 'processing' : 'queued' + started_at = submission_processing_file_timestamp + elsif state == 'ready' && !submission_pdf_ready? + state = 'failed' + end + + if %w[queued processing].include?(state) && submission_processing_timed_out?(now: now, started_at: started_at) + 'timed_out' + else + state + end + end + + def submission_processing_timed_out?(now: Time.current, started_at: submission_processing_started_at) + started_at ||= submission_processing_file_timestamp + started_at.present? && started_at < now - submission_processing_timeout + end + + # A retry or regeneration that restores the done archive is waiting or + # running. It stages nothing until its job starts, and any upload accepted + # meanwhile would be replaced by that archive. + def submission_archive_restore_pending?(now: Time.current) + %w[retry_archive regenerate_only].include?(submission_processing_mode) && + %w[queued processing].include?(effective_submission_processing_state(now: now)) + end + + def submission_processing_retryable?(now: Time.current) + %w[failed timed_out].include?(effective_submission_processing_state(now: now)) && submission_retry_source_available? + end + + def submission_processing_snapshot(now: Time.current) + state = effective_submission_processing_state(now: now) + { + has_pdf: state == 'ready' && submission_pdf_ready?, + pdf_ready: submission_pdf_ready?, + submission_files_ready: submission_files_ready?, + processing_pdf: %w[queued processing].include?(state), + processing_state: state, + processing_started_at: submission_processing_started_at, + processing_finished_at: submission_processing_finished_at, + retryable: submission_processing_retryable?(now: now), + poll_after_seconds: %w[queued processing].include?(state) ? 3 : nil + } + end + + # processing_mode is recorded whenever an attempt is queued. The uploader, + # test_submission and accepted_tii_eula are recorded when a new upload is + # queued, and left as they are when a retry or regeneration is queued. + def mark_submission_processing!(state, error_code: nil, now: Time.current, processing_mode: nil, user_id: nil, test_submission: nil, accepted_tii_eula: nil) + raise ArgumentError, "Unknown submission processing state: #{state}" unless SUBMISSION_PROCESSING_STATES.include?(state.to_s) + + processing_task = submission_processing_task + + attributes = { + submission_processing_state: state, + submission_processing_error_code: error_code, + submission_processing_finished_at: %w[ready failed].include?(state.to_s) ? now : nil + } + if state.to_s == 'queued' + attributes[:submission_processing_started_at] = now + attributes[:submission_processing_attempts] = processing_task.submission_processing_attempts.to_i + 1 + attributes[:submission_processing_mode] = processing_mode unless processing_mode.nil? + attributes[:submission_processing_user_id] = user_id unless user_id.nil? + attributes[:submission_processing_test_submission] = test_submission unless test_submission.nil? + attributes[:submission_processing_accepted_tii_eula] = accepted_tii_eula unless accepted_tii_eula.nil? + end + + Task.transaction do + processing_task.submission_processing_targets.sort_by(&:id).each do |target| + target.update!(attributes) + end + end + end + + def retry_submission_processing!(user) + processing_task = submission_processing_task + submission_processing_lock_target.with_lock do + processing_task.reload + unless processing_task.submission_processing_retryable? + raise ArgumentError, 'This submission is not ready to retry.' + end + + processing_mode = if processing_task.submission_processing_mode == 'regenerate_only' + # A failed regeneration is retried as a regeneration. + # Its upload's side effects already ran. + 'regenerate_only' + elsif !processing_task.folder_exists_in_new? || processing_task.folder_exists_in_process? + 'retry_archive' + else + 'process' + end + processing_task.enqueue_submission_processing!(user, processing_mode: processing_mode) + end + end + + def regenerate_submission!(user) + processing_task = submission_processing_task + submission_processing_lock_target.with_lock do + processing_task.reload + if %w[queued processing].include?(processing_task.effective_submission_processing_state) + raise ArgumentError, 'A submission is already being processed.' + end + + processing_task.enqueue_submission_processing!(user, processing_mode: 'regenerate_only') + end + end + + def submission_processing_timeout + configured_seconds = ENV.fetch('DF_SUBMISSION_PROCESSING_TIMEOUT_SECONDS', SUBMISSION_PROCESSING_TIMEOUT.to_i).to_i + configured_seconds = SUBMISSION_PROCESSING_TIMEOUT.to_i unless configured_seconds.positive? + configured_seconds.seconds + end + + def submission_processing_file_timestamp + processing_task = submission_processing_task + paths = [processing_task.student_work_dir(:new, false), processing_task.student_work_dir(:in_process, false)].select { |path| Dir.exist?(path) } + paths.filter_map do |path| + File.mtime(path) + rescue SystemCallError + nil + end.min + end + + def submission_processing_task + return self unless group_submission + + group_submission.submitter_task || self + end + + def submission_processing_lock_target + group_submission&.group || (group if group_task?) || submission_processing_task + end + + def submission_processing_targets + processing_task = submission_processing_task + return [processing_task] unless processing_task.group_submission + + processing_task.group_submission.tasks.to_a.presence || [processing_task] + end + + def submission_retry_source_available? + submission_files_ready? || (folder_exists_in_new? && !folder_exists_in_process?) + end + + def enqueue_submission_processing!(user, processing_mode:) + if %w[retry_archive regenerate_only].include?(processing_mode) && !submission_files_ready? + raise 'The submitted files are no longer available.' + end + + mark_submission_processing!('queued', processing_mode: processing_mode) + # For a group the new attempt was written through other instances. + reload + + # Replay the upload this attempt belongs to: the same uploader and the same + # options, so a test submission stays a test submission. Turnitin consent + # only travels with the uploader who gave it. If that account is gone the + # caller is used without it. The attempt number lets the job stand down if + # a newer upload is accepted first. + uploader = User.find_by(id: submission_processing_user_id) + job_id = AcceptSubmissionJob.perform_async( + id, + (uploader || user).id, + uploader.present? && submission_processing_accepted_tii_eula, + submission_processing_test_submission, + processing_mode, + submission_processing_attempts + ) + raise ArgumentError, 'Submission processing is already queued or running. Check again shortly.' if job_id.blank? + + job_id + end + protected :enqueue_submission_processing! + + # Restore the immutable done archive to a fresh staging directory, then swap + # it into `new` only after the complete archive has been extracted. Existing + # `new` and `in_process` work is retained until that point and restored if the + # swap itself fails. + def prepare_submission_regeneration! + processing_task = submission_processing_task + # Compare records, not objects: GroupSubmission#submitter_task loads a new + # instance each time, so object identity never matches and this recursed. + return processing_task.prepare_submission_regeneration! unless processing_task == self + + zip_path = zip_file_path_for_done_task + has_zip = zip_path.present? && File.file?(zip_path) + raise 'The submitted files are no longer available.' unless has_zip || uncompressed_done_files? + + new_path = student_work_dir(:new, false).delete_suffix(File::SEPARATOR) + in_process_path = student_work_dir(:in_process, false).delete_suffix(File::SEPARATOR) + suffix = "submission-retry-#{SecureRandom.hex(8)}" + staging_path = "#{new_path}.#{suffix}" + new_backup = "#{new_path}.#{suffix}.backup" + in_process_backup = "#{in_process_path}.#{suffix}.backup" + + if has_zip + extract_preserved_submission!(zip_path, staging_path) + else + # 11.0.x regenerated these through move_done_to_new, so keep them working. + copy_preserved_submission!(student_work_dir(:done, false), staging_path) + end + + new_was_present = File.exist?(new_path) || File.symlink?(new_path) + in_process_was_present = File.exist?(in_process_path) || File.symlink?(in_process_path) + new_backed_up = false + in_process_backed_up = false + swapped = false + begin + if new_was_present + FileUtils.mv(new_path, new_backup) + new_backed_up = true + end + if in_process_was_present + FileUtils.mv(in_process_path, in_process_backup) + in_process_backed_up = true + end + FileUtils.mv(staging_path, new_path) + swapped = true + rescue StandardError + # Only clear new_path when the original was moved aside (or never + # existed). If the first backup move failed, new_path still holds the + # original files and must be left alone. + if (new_backed_up || !new_was_present) && (File.exist?(new_path) || File.symlink?(new_path)) + FileUtils.rm_rf(new_path) + end + FileUtils.mv(new_backup, new_path) if new_backed_up + FileUtils.mv(in_process_backup, in_process_path) if in_process_backed_up + raise + ensure + FileUtils.rm_rf(staging_path) + if swapped + FileUtils.rm_rf(new_backup) + FileUtils.rm_rf(in_process_backup) + end + end + + true + end + + def extract_preserved_submission!(zip_path, staging_path) + expected_prefix = "#{id}/" + extracted_files = 0 + staging_root = File.expand_path(staging_path) + + Zip::File.open(zip_path) do |zip| + zip.each do |entry| + next unless entry.name.start_with?(expected_prefix) + + relative_name = entry.name.delete_prefix(expected_prefix) + next if relative_name.blank? + + destination = File.expand_path(relative_name, staging_root) + unless destination.start_with?("#{staging_root}#{File::SEPARATOR}") + raise 'The preserved submission archive contains an unsafe path.' + end + + if entry.directory? + FileUtils.mkdir_p(destination) + else + FileUtils.mkdir_p(File.dirname(destination)) + entry.extract(destination) { true } + extracted_files += 1 + end + end + end + + raise 'The preserved submission archive is empty or invalid.' if extracted_files.zero? + rescue StandardError + FileUtils.rm_rf(staging_path) + raise + end + private :extract_preserved_submission! + + # The uncompressed done folder holds the same files a done zip holds under + # its task id. Links are skipped so nothing outside the folder is copied. + def copy_preserved_submission!(done_dir, staging_path) + FileUtils.mkdir_p(staging_path) + Dir.children(done_dir).each do |name| + source = File.join(done_dir, name) + next if File.symlink?(source) + + FileUtils.cp_r(source, staging_path) + end + + raise 'The preserved submission folder is empty.' if Dir.empty?(staging_path) + rescue StandardError + FileUtils.rm_rf(staging_path) + raise + end + private :copy_preserved_submission! + + # The time zone this task's deadlines are read in. + # + # A deadline written as a day belongs to the student's day, so the zone comes + # from the campus the student is enrolled at. Campus#timezone already falls + # back to the application zone when the column is not set, and a project with + # no campus falls back to the same place, so an install that has not filled in + # campus time zones behaves exactly as it did before. Nothing here depends on + # config.time_zone being set to anything in particular. + def deadline_time_zone + name = project&.campus&.timezone + zone = ActiveSupport::TimeZone[name] if name.present? + + zone || Time.zone + end + + # The calendar day a deadline falls on, read in this task's own zone. + # + # Reading it in whatever zone the value happened to be loaded in is what let + # the day move. A campus on Australian time changes its offset from UTC by an + # hour twice a year, so the same wall clock deadline sat on one UTC day in + # summer and the next one in winter, and every date built from those parts + # drifted with it. + def deadline_date(value) + value.in_time_zone(deadline_time_zone).to_date + end + # Get the raw extension date - with extensions representing weeks def raw_extension_date - target_date.to_date + extensions.weeks + deadline_date(target_date) + extensions.weeks end def max_date_with_spec_con_days - task_definition.due_date.to_date + project.spec_con_days.days + deadline_date(task_definition.due_date) + project.spec_con_days.days end # Get the adjusted extension date, which ensures it is never past the due date @@ -344,9 +772,52 @@ def apply_for_extension(user, text, weeks) end end + notify_extension_request_recipient(extension, user) + extension end + # Tell whoever has to assess a student's extension request that it arrived. + # + # extension.recipient is already worked out above: the task's tutor, which + # tutor_for makes the main convenor when there is no tutor, or the main + # convenor when more weeks are asked for than are left before the due date. + # Do not recalculate it. + # + # Only a request still waiting on a person notifies. Staff creating an + # extension and a unit approving it automatically both assess it straight + # away, and there is nothing left for the tutor to do. Checking assessed? + # rather than those two conditions also covers an automatic approval that + # failed, which leaves the request waiting. + # + # The student's reason is left out, the same way comment text is. Failures + # are logged and swallowed so the request itself is never lost. + def notify_extension_request_recipient(extension, user) + # The same test role_for uses for :student, without its staff query. + return unless user == project.student + return if extension.assessed? + + recipient = extension.recipient + return if recipient.blank? || recipient == user + + product_name = Doubtfire::Application.config.institution[:product_name] + + # 'task', not 'extension': 'extension' has no preference behind it, and a + # tutor must be able to switch this off with their other task + # notifications. The student's extension_assessed keeps 'extension'. + NotificationService.notify( + user: recipient, + type: 'task', + event: 'extension_requested', + message: "#{user.name} asked for an extension on #{task_definition.name} in #{product_name}.", + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}", + notifiable: extension + ) + rescue StandardError => e + logger.error "Failed to raise extension_requested notification for task #{id}: #{e.message}" + end + private :notify_extension_request_recipient + def weeks_can_extend deadline = max_date_with_spec_con_days current_due = raw_extension_date @@ -364,6 +835,7 @@ def grant_extension(by_user, weeks) return false unless weeks_to_extend > 0 if update(extensions: self.extensions + weeks_to_extend) + association(:granted_extension_comments).reset # Was the task previously assessed as time exceeded? ... with the extension should this change? if self.task_status == TaskStatus.time_exceeded && submitted_before_due? update(task_status: TaskStatus.ready_for_feedback) @@ -376,6 +848,178 @@ def grant_extension(by_user, weeks) end end + # + # The effective resubmission deadline + # + # When staff send a task back for more work the student needs time to do that + # work, so a task whose deadline is close is extended by the unit's + # resubmission extension. The rule itself is the four methods below, so a + # change to the rule is a change in one place. + # + # See docs/submission-lifecycle/effective-resubmission-deadline.md + # + + # The statuses that hand a task back to the student for more work + def resubmission_extension_statuses + [TaskStatus.fix_and_resubmit, TaskStatus.discuss, TaskStatus.rediscuss, TaskStatus.demonstrate] + end + + # How close the deadline has to be before a resubmission earns an extension + def resubmission_extension_window + 7.days + end + + # How many weeks the unit adds when a resubmission earns an extension + def resubmission_extension_weeks + unit.extension_weeks_on_resubmit_request + end + + # The moment this task's deadline actually passes. + # + # A deadline set as a day runs to the end of that day anywhere on earth, and + # which day that is is read in the task's own zone. This is the "effective + # deadline" the ticket is named after and it is the one value the window, the + # late check and the interface should all agree on. + def effective_deadline + to_same_day_anywhere_on_earth(effective_deadline_date) + end + + # Share the same calendar day with the task API and existing subscription feed. + def effective_deadline_date + deadline_date(local_due_date) + end + + def effective_deadline_source + granted_extension_comments.first + end + + def effective_deadline_source_id + effective_deadline_source&.id if extensions.positive? && !unit.allow_flexible_dates + end + + def effective_deadline_reason + return 'flexible_date' if unit.allow_flexible_dates + return 'standard_due_date' unless extensions.positive? + + effective_deadline_source&.resubmission_extension? ? 'post_feedback_extension' : 'approved_extension' + end + + # The far edge of the window: seven calendar days after this assessment, in + # the task's own zone. + # + # The window is added as a duration to a time in that zone, so it lands at the + # same wall clock seven days later even when the clocks change in between. The + # week Melbourne moves onto daylight saving is 167 real hours long and the + # week it moves off is 169, and counting either as a flat 168 moved the edge of + # the window by an hour. + def resubmission_extension_window_end(assess_date = Time.zone.now) + assess_date.in_time_zone(deadline_time_zone) + resubmission_extension_window + end + + # Is the deadline close enough, at the moment of this assessment, for the + # resubmission extension to apply? The assessment's own time is used rather + # than the wall clock, so that reprocessing an event gives the answer it gave + # when it happened, and so dependent tasks fixed recursively are judged at the + # same moment as the task that triggered them. + # + # Both sides of this comparison are resolved in the task's own zone rather + # than in whatever the application zone happens to be, so the answer does not + # depend on config.time_zone being set. + def resubmission_extension_window_open?(assess_date = Time.zone.now) + effective_deadline < resubmission_extension_window_end(assess_date) + end + + # The resubmission extension recorded for the current round of feedback, or + # nil if this round has not earned one. A round starts when the + # student submits, which is the same signal times_assessed uses, so a genuine + # resubmission earns a new extension while a repeated assessment, a re-save or + # a duplicate event does not. + def resubmission_extension_comment + return nil if submission_date.nil? + + comments + .where(type: 'ExtensionComment') + .where.not(task_status_id: nil) + .where('date_extension_assessed >= ?', submission_date) + .order(:id) + .last + end + + # Apply the resubmission extension for this assessment, if the rule calls for + # one and this round of feedback has not already had one. Returns the comment + # recording the extension, or nil when no extension was applied. + def grant_resubmission_extension(status, by_user, assess_date = Time.zone.now) + return nil unless resubmission_extension_statuses.include?(status) + + extension = nil + notification = nil + self.class.transaction do + # Assess sets unsaved status fields before calling this method. Lock a + # separate instance so those fields are preserved, and refresh the value + # that a concurrent assessment may already have changed. + persisted_task = self.class.lock.find(id) + self.extensions = persisted_task.extensions + self.submission_date = persisted_task.submission_date if persisted_task.submission_date.present? + task_definition.reload + next unless task_definition.resubmission_extensions_enabled + next unless resubmission_extension_weeks.positive? && !unit.allow_flexible_dates + next unless can_apply_for_extension? && resubmission_extension_window_open?(assess_date) + next if resubmission_extension_comment.present? + + previous_deadline = effective_deadline + weeks = [resubmission_extension_weeks, weeks_can_extend].min + next unless grant_extension(by_user, weeks) + + # The extension and its replay marker must either both commit or neither. + extension = record_resubmission_extension(status, by_user, assess_date, weeks) + if effective_deadline > previous_deadline + notification = NotificationService.reserve( + user: project.student, + type: 'extension', + event: 'resubmission_deadline_changed', + message: "Your task deadline is now #{effective_deadline_date.iso8601} (end of day anywhere on earth) after feedback requiring further action. Open OnTrack for details.", + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}", + dedupe_key: "resubmission-deadline:#{extension.id}", + notifiable: extension + ) + end + end + if notification + ActiveRecord.after_all_transactions_commit do + deliver_resubmission_deadline_notification(notification) + end + end + extension + end + + def deliver_resubmission_deadline_notification(notification) + NotificationService.deliver(notification) + rescue StandardError => e + Rails.logger.error "Failed to deliver resubmission deadline notification for task #{id}: #{e.class}" + end + private :deliver_resubmission_deadline_notification + + # Record why the deadline moved and which assessment moved it, so the + # interface and the notifications can explain the change, and so a repeat of + # the same assessment can see that it has already been handled. + def record_resubmission_extension(status, by_user, assess_date, weeks) + extension = ExtensionComment.new + extension.task = self + extension.user = by_user + extension.recipient = by_user == project.student ? tutor : project.student + extension.content_type = :extension + extension.task_status = status + extension.assessor = by_user + extension.extension_weeks = weeks + extension.extension_granted = true + extension.date_extension_assessed = assess_date + extension.comment = "**Automated Message:** This task was set to #{status.name} within a week of its deadline, so it was extended by #{weeks} #{'week'.pluralize(weeks)} to give you time to resubmit." + extension.extension_response = "Time extended to #{due_date.strftime('%a %b %e')}" + extension.save! + + extension + end + # Applying for a scorm extension will create a scorm extension comment def apply_for_scorm_extension(user, text) extension = ScormExtensionComment.create @@ -605,6 +1249,10 @@ def trigger_transition(trigger: '', by_user: nil, bulk: false, group_transition: # State transitions based upon the trigger # + # Remember the status before the transition so we can tell, at the end, + # whether it actually changed. An unchanged status must not notify (EN-E02). + status_id_before_transition = task_status_id + status = TaskStatus.status_for_name(trigger) case status @@ -676,9 +1324,95 @@ def trigger_transition(trigger: '', by_user: nil, bulk: false, group_transition: end end + # Tell the responsible tutor when a student submits for marking (EN-V06) + # or asks for help. + notify_tutor_of_student_request(by_user, role, status_id_before_transition, group_transition) + + # EN-E02: tell the student when a staff member changed their task's status. + notify_student_of_status_change(by_user, role, status_id_before_transition) + true end + # Tell the responsible tutor when a student's task genuinely moves into a + # state that asks the tutor to act: ready for feedback (task_submitted) or + # need help (task_help_requested). A student's other statuses need nothing + # from the tutor, so they stay silent. EN-E02 shares this transition seam, + # but its tutor-only role guard is deliberately disjoint from this + # student-only one, so one transition cannot raise both events. + # + # A group submission fans the same transition out to every member task. Only + # the original action notifies; internal group transitions are suppressed so + # one logical submission cannot amplify into duplicate tutor emails. + def notify_tutor_of_student_request(by_user, role, previous_status_id, group_transition) + return unless [:student, :group_member].include?(role) + return if group_transition + return if task_status_id == previous_status_id + + product_name = Doubtfire::Application.config.institution[:product_name] + + case task_status + when TaskStatus.ready_for_feedback + event = 'task_submitted' + action = "submitted #{task_definition.name} for marking" + when TaskStatus.need_help + event = 'task_help_requested' + action = "asked for help with #{task_definition.name}" + else + return + end + + recipient = project&.tutor_for(task_definition) + student = project&.student + return if recipient.blank? || student.blank? || recipient == by_user + + NotificationService.notify( + user: recipient, + type: 'task', + event: event, + message: "#{student.name} #{action} in #{product_name}.", + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}", + notifiable: self + ) + rescue StandardError => e + logger.error "Failed to raise #{event || 'tutor'} notification for task #{id}: #{e.message}" + end + + # Tell the student that a staff member changed the status of their task. + # + # Only a tutor's action notifies (role == :tutor); a student changing their + # own task must never email themselves. And only a real change notifies: an + # unchanged status is a no-op. + # + # The new status value is deliberately kept out of the notification, the same + # way the comment text is in notify_comment_recipient. The email is a prompt to + # come back to OnTrack, not a copy of the result. + # + # Raising a notification must never roll back the transition, so failures are + # logged and swallowed. NotificationService already rescues mail errors; this + # catches the record write and anything else unexpected. + def notify_student_of_status_change(by_user, role, previous_status_id) + return unless role == :tutor + return if task_status_id == previous_status_id + + recipient = project&.student + # recipient == by_user is belt and braces: once role == :tutor the actor + # cannot be the student, since user_role checks user == student first. Kept + # so a future change to user_role cannot start emailing someone themselves. + return if recipient.blank? || recipient == by_user + + NotificationService.notify( + user: recipient, + type: 'task', + event: 'task_status_changed', + message: "#{by_user.name} updated the status of #{task_definition.abbreviation} in #{unit.code}.", + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}", + notifiable: self + ) + rescue StandardError => e + logger.error "Failed to raise task_status_changed notification for task #{id}: #{e.message}" + end + def has_discussed_in_class_comment? comments.where(content_type: 'discussed_in_class').exists? end @@ -690,7 +1424,7 @@ def has_manual_feedback_since_first_ready_for_feedback? .pick(:created_at) feedback_comments = comments - .where(content_type: %w[text audio image pdf discussion]) + .where(content_type: %w[text audio image pdf discussion document spreadsheet]) .where(user_id: unit.staff.select(:user_id)) feedback_comments = feedback_comments.where('created_at >= ?', first_ready_for_feedback_at) if first_ready_for_feedback_at @@ -700,7 +1434,7 @@ def has_manual_feedback_since_first_ready_for_feedback? def has_recent_manual_feedback_from_tutor?(tutor) comments - .where(content_type: %w[text audio image pdf discussion]) + .where(content_type: %w[text audio image pdf discussion document spreadsheet]) .where(user: tutor) .where('created_at >= ?', 10.minutes.ago) .where.not("COALESCE(comment, '') LIKE ?", '**Automated Message:%') @@ -780,16 +1514,12 @@ def assess(task_status, assessor, assess_date = Time.zone.now, recursive_fix = f self.completion_date = assess_date if completion_date.nil? else self.completion_date = nil - - # Grant an extension on fix if due date is within 1 week - case task_status - when TaskStatus.fix_and_resubmit, TaskStatus.discuss, TaskStatus.rediscuss, TaskStatus.demonstrate - if to_same_day_anywhere_on_earth(due_date) < Time.zone.now + 7.days && can_apply_for_extension? && unit.extension_weeks_on_resubmit_request > 0 - grant_extension(assessor, unit.extension_weeks_on_resubmit_request) - end - end end + # All declared feedback outcomes must reach the eligibility check, including + # Discuss/Demonstrate, which also mark the work ready for completion. + grant_resubmission_extension(task_status, assessor, assess_date) + # Save the task if save! if assessor == tutor && task_status != TaskStatus.time_exceeded && task_status != TaskStatus.assess_in_portfolio @@ -926,9 +1656,13 @@ def weight task_definition.weighting.to_f end - def add_text_comment(user, text, reply_to_id = nil) + def add_text_comment(user, text, reply_to_id = nil, client_request_id = nil) + return nil if user.nil? || text.nil? + + # Strip before the emptiness check, as 11.0.x did: strip also removes NUL, + # which blank? does not count as whitespace. text = text.strip - return nil if user.nil? || text.nil? || text.empty? + return nil if text.blank? lc = comments.last @@ -944,11 +1678,55 @@ def add_text_comment(user, text, reply_to_id = nil) comment.content_type = :text comment.recipient = user == project.student ? project.tutor_for(task_definition) : project.student comment.reply_to_id = reply_to_id + comment.client_request_id = client_request_id comment.save! + notify_comment_recipient(comment) + comment end + # Tell the other party that a comment arrived. + # + # comment.recipient is already worked out above: the tutor when a student + # commented, the student when a tutor commented. Do not recalculate it. + # + # A project with no tutor for this task definition has no recipient, so the + # guard is required and not defensive padding. + # + # The comment text is deliberately not put in the notification. The email is a + # prompt to come back to OnTrack, not a copy of the conversation. + # + # Raising a notification must never stop a comment being posted, so failures + # are logged and swallowed. NotificationService already rescues mail errors; + # this catches the record write and anything else unexpected. + def notify_comment_recipient(comment) + return if comment.recipient.blank? + + # An automated comment is stored against the tutor because a comment needs an + # author, but saying they wrote it is not true and reads as a person having + # replied. Name OnTrack instead, and use its own event so a client can tell + # the two apart without matching on the text. + automated = comment.respond_to?(:automated?) && comment.automated? + message = + if automated + "OnTrack added a note to #{task_definition.abbreviation} in #{unit.code}." + else + "#{comment.user.name} commented on #{task_definition.abbreviation} in #{unit.code}." + end + + NotificationService.notify( + user: comment.recipient, + type: 'feedback', + event: automated ? 'task_automated_comment_created' : 'task_comment_created', + message: message, + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}/feedback", + notifiable: comment + ) + rescue StandardError => e + logger.error "Failed to raise task_comment_created notification for task #{id}: #{e.message}" + end + def individual_task_or_submitter_of_group_task? return true if !group_task? # its individual return true if group.blank? # no group yet... so individual @@ -973,10 +1751,10 @@ def add_status_comment(current_user, status) def add_discussed_comment(current_user) comment = 'Discussed in class' - lc = comments.last - - # don't add if duplicate comment - return if lc && lc.user == current_user && lc.content_type == 'discussed_in_class' && lc.comment == comment + # This comment represents a boolean task state, so an intervening feedback + # comment must not allow a second marker to be created. + existing = comments.where(content_type: 'discussed_in_class').last + return existing if existing discussed = TaskDiscussedComment.create discussed.task = self @@ -987,6 +1765,14 @@ def add_discussed_comment(current_user) discussed end + # Undo a "discussed in class" mark by removing every marker on this task. + # Legacy data can contain duplicates separated by ordinary feedback comments. + # destroy_all is intentional so TaskComment callbacks and dependent read + # receipt destruction still run for every marker. + def remove_discussed_comment + comments.where(content_type: 'discussed_in_class').destroy_all + end + def add_checked_in_comment(current_user) discussed = TaskCheckedInComment.create discussed.task = self @@ -1014,33 +1800,72 @@ def add_discussion_comment(user, prompts) end discussion.mark_as_read(user, unit) + notify_discussion_request_recipient(discussion) logger.info(discussion) return discussion end + # EN-V08 was originally described as a discussion booking notification, but + # OnTrack has no booking or appointment record to hook. A discussion comment + # is the point where a tutor actually raises an audio prompt for a student, + # so notify the student once that prompt and its attachments are ready. + # + # Prompt content is deliberately left out of the notification and email. A + # notification failure must not stop the discussion comment being created. + def notify_discussion_request_recipient(discussion) + return if discussion.recipient.blank? + + NotificationService.notify( + user: discussion.recipient, + type: 'feedback', + event: 'discussion_request_created', + message: 'A discussion prompt is ready for you.', + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}/feedback", + notifiable: discussion + ) + rescue StandardError => e + logger.error "Failed to raise discussion_request_created notification for task #{id}: #{e.message}" + end + private :notify_discussion_request_recipient + # TODO: Refactor to attachment comment (with inheritance on model) - def add_comment_with_attachment(user, tempfile, reply_to_id = nil) + def add_comment_with_attachment(user, tempfile, reply_to_id = nil, text = nil, client_request_id = nil) ensured_group_submission if group_task? && group - comment = TaskComment.create - comment.task = self - comment.user = user - comment.reply_to_id = reply_to_id - if FileHelper.accept_file(tempfile, "comment attachment audio test", "audio")[:accepted] - comment.content_type = :audio - elsif FileHelper.accept_file(tempfile, "comment attachment image test", "image")[:accepted] - comment.content_type = :image - elsif FileHelper.accept_file(tempfile, "comment attachment pdf", "document")[:accepted] - comment.content_type = :pdf - else - raise "Unknown comment attachment type" + validation = FileHelper.accept_file(tempfile, 'comment attachment', 'comment_attachment') + return nil unless validation[:accepted] + + attachment_type = validation[:category] + + comment = TaskComment.new( + task: self, + user: user, + recipient: user == project.student ? project.tutor_for(task_definition) : project.student, + reply_to_id: reply_to_id, + comment: text.presence, + content_type: attachment_type, + client_request_id: client_request_id + ) + + TaskComment.transaction do + # Allocate the id inside the transaction because the attachment storage + # path is id-based. Any conversion/storage failure rolls the row and its + # read receipt back together. + comment.save! + begin + raise 'Error attaching uploaded file.' unless comment.add_attachment(tempfile) + rescue StandardError + # Filesystem operations are not transactional, and the rollback resets + # the new row's id that the storage path is built from. Remove any + # moved or converted file now, while that path is still known. + FileUtils.rm_f(comment.attachment_path) if comment.attachment_extension.present? + raise + end end - comment.recipient = user == project.student ? project.tutor_for(task_definition) : project.student - raise "Error attaching uploaded file." unless comment.add_attachment(tempfile) - - comment.save! + # Notify once after the attachment transaction succeeds. + notify_comment_recipient(comment) comment end @@ -1058,6 +1883,13 @@ def add_feedback_review_request_comment(current_user) request.comment = comment request.recipient = current_user == project.student ? project.tutor_for(task_definition) : project.student request.save! + + # A feedback review request is a comment the recipient needs to act on just + # like a text or attachment comment, and both of those notify. This path + # stayed silent, so a student's request for a review reached the tutor as no + # email, push or in-app notification at all. Notify here too. + notify_comment_recipient(request) + request end @@ -1155,6 +1987,7 @@ def compress_new_to_done(task_dir: student_work_dir(:new, false), zip_file_path: raise "Multiple team member submissions received at the same time. Please ensure that only one member submits the task." if group_task? && self != group_submission.submitter_task zip_file = zip_file_path || zip_file_path_for_done_task + temp_zip = nil return false if zip_file.nil? || (!Dir.exist? task_dir) # compress image files - convert to jpg @@ -1168,7 +2001,7 @@ def compress_new_to_done(task_dir: student_work_dir(:new, false), zip_file_path: FileUtils.rm("#{task_dir}#{img}") unless dest_file == "#{task_dir}#{img}" end - input_files = Dir.entries(task_dir).select { |f| (f =~ /^\d{3}.(cover|document|code|image|zip|archive)/) == 0 } + input_files = Dir.entries(task_dir).select { |f| (f =~ /^\d{3}.(cover|document|code|image|zip|archive|csv)/) == 0 } if input_files.length != task_definition.number_of_uploaded_files logger.error "Error processing task #{log_details} - missing files expected #{task_definition.number_of_uploaded_files} got #{input_files.length}" @@ -1178,14 +2011,17 @@ def compress_new_to_done(task_dir: student_work_dir(:new, false), zip_file_path: logger.info "Creating new zip file for task #{id} in #{zip_file}" - # We have what looks like a good submission, remove old zip - FileUtils.rm_f(zip_file) - # copy all files into zip zip_dir = File.dirname(zip_file) FileUtils.mkdir_p zip_dir - Zip::File.open(zip_file, Zip::File::CREATE) do |zip| + # Build the new archive alongside the existing done zip and swap it in only + # once it has closed cleanly. Writing straight over zip_file, after removing + # it first, meant a failed add left the task with no readable submission at + # all, having already destroyed the previously accepted one. + temp_zip = "#{zip_file}.tmp-#{SecureRandom.hex(8)}" + + Zip::File.open(temp_zip, Zip::File::CREATE) do |zip| zip.mkdir id.to_s input_files.each do |in_file| final_name = in_file @@ -1198,8 +2034,25 @@ def compress_new_to_done(task_dir: student_work_dir(:new, false), zip_file_path: zip.add "#{id}/#{final_name}", "#{task_dir}#{in_file}" end end + + # The archive is complete on disk, so it is now safe to swap it in. File.rename + # is an atomic same-directory replace and, unlike FileUtils.mv(force: true), + # raises if it fails instead of silently leaving the old zip in place while we + # go on to delete the source and report success. + File.rename(temp_zip, zip_file) + temp_zip = nil ensure - FileUtils.rm_rf(task_dir) if rm_task_dir + if temp_zip + # We entered the archive-write phase but did not swap the new zip in, so + # the write or the rename failed. Keep the source files in task_dir so the + # previously accepted submission can be recovered, and remove only the + # half-written temporary archive. + FileUtils.rm_f(temp_zip) + elsif rm_task_dir + # A clean success, an early rejection (missing files), or the group-guard + # raise: discard the source files as before. + FileUtils.rm_rf(task_dir) + end end true @@ -1600,8 +2453,11 @@ def create_submission_and_trigger_state_change(user, propagate = true, contribut # def accept_submission(current_user, files, ui, contributions, trigger, alignments, accepted_tii_eula: false, test_submission: false) submission_lock_target.with_lock do - # Ensure there is not a submission already in process - if processing_pdf? + # Ensure there is not a submission already in process. A retry or + # regeneration that restores the done archive stages no files until its + # job starts, so the folder check cannot see it, and that job would then + # replace this upload with the previous archive. + if processing_pdf? || submission_processing_task.submission_archive_restore_pending? ui.error!({ 'error' => 'A submission is already being processed. Please wait for the current submission process to complete.' }, 403) end @@ -1707,7 +2563,16 @@ def accept_submission(current_user, files, ui, contributions, trigger, alignment logger.info "Submission accepted! Status for task #{id} is now #{trigger}" # Trigger processing of new submission - async - AcceptSubmissionJob.perform_async(id, current_user.id, accepted_tii_eula, test_submission) + processing_task = submission_processing_task + processing_task.mark_submission_processing!( + 'queued', + processing_mode: 'process', + user_id: current_user.id, + test_submission: test_submission.present?, + accepted_tii_eula: accepted_tii_eula.present? + ) + job_id = AcceptSubmissionJob.perform_async(processing_task.id, current_user.id, accepted_tii_eula, test_submission, false) + processing_task.mark_submission_processing!('failed', error_code: 'queue_conflict') if job_id.blank? end end @@ -1833,9 +2698,17 @@ def delete_associated_files end end - # Use the current DateTime to calculate a new DateTime for the last moment of the same - # day anywhere on earth + # The last moment of the same day anywhere on earth. + # + # A deadline set as a day is not over until that day is over everywhere, which + # is 23:59:59 at UTC-12. Which day that is has to be read in the task's own + # zone, because a timestamp near midnight belongs to different calendar days + # in different zones. This used to read the day, month and year straight off + # the value as it happened to be loaded, so the answer moved by a whole day + # when a campus changed its offset for daylight saving. The result is built at + # a fixed -12:00 offset, which never observes daylight saving itself. def to_same_day_anywhere_on_earth(date) - DateTime.new(date.year, date.month, date.day, 23, 59, 59, '-12:00') + day = deadline_date(date) + Time.new(day.year, day.month, day.day, 23, 59, 59, '-12:00') end end diff --git a/app/models/task_definition.rb b/app/models/task_definition.rb index 7ef377811f..b1dd6cb3c0 100644 --- a/app/models/task_definition.rb +++ b/app/models/task_definition.rb @@ -57,6 +57,11 @@ def self.permissions before_destroy :delete_associated_files + # The database default protects rows written by an older application process + # during a rolling deployment. Current code explicitly keeps new definitions + # untracked until a supported creation workflow has fully configured them. + before_create :defer_new_task_notifications + after_update :move_files_on_abbreviation_change, if: :saved_change_to_abbreviation? after_update :remove_old_group_submissions, if: :has_removed_group? after_update :check_and_update_tii_status, if: :saved_change_to_upload_requirements? @@ -65,12 +70,15 @@ def self.permissions after_update :reset_overdue_tasks, if: :saved_change_to_due_date? # Model associations + validates :resubmission_extensions_enabled, inclusion: { in: [true, false] } + belongs_to :unit, optional: false # Foreign key belongs_to :group_set, optional: true belongs_to :tutorial_stream, optional: true belongs_to :overseer_image, optional: true - has_many :tasks, dependent: :destroy # Destroying a task definition will also nuke any instances + has_many :tasks, dependent: :destroy # Destroying a task definition will also nuke any instances + has_many :peer_progress_snapshots, dependent: :destroy, inverse_of: :task_definition has_many :group_submissions, dependent: :destroy # Destroying a task definition will also nuke any group submissions has_many :learning_outcomes, as: :context, dependent: :destroy has_many :overseer_steps, -> { order(:sort_order) }, inverse_of: :task_definition, dependent: :destroy @@ -133,6 +141,24 @@ def grade_start_date(target_grade) grade_due_dates.find { |g| g.target_grade == target_grade.to_i }&.start_date end + # Opt this fully configured definition into immediate and scheduled + # availability notifications. Existing definitions are backfilled from the + # rollout time by the migration; new supported workflows use the unit start + # so already-available copied/imported tasks can still be announced. + def enable_new_task_notifications! + notification_start = [unit.start_date, start_date].compact.min || Time.current + # Legacy definitions may fail unrelated validations; this internal marker + # must still be repairable by a retrying notification job. + # rubocop:disable Rails/SkipsModelValidations + update_column(:new_task_notifications_from, notification_start) + # rubocop:enable Rails/SkipsModelValidations + end + + def defer_new_task_notifications + self.new_task_notifications_from = nil + new_task_notifications_from_will_change! + end + def unit_must_be_same if unit.present? and tutorial_stream.present? and not unit.eql? tutorial_stream.unit errors.add(:unit, "should be same as the unit in the associated tutorial stream") @@ -174,6 +200,7 @@ def check_existing_prerequisites # Copy this task into the other unit def copy_to(other_unit) new_td = self.dup + new_td.new_task_notifications_from = nil # change the unit... new_td.unit_id = other_unit.id # for database @@ -341,7 +368,7 @@ def check_upload_requirements_format end # Check the type is either document, image, code, or zip - unless %w(document image code zip).include? req['type'] + unless %w(document image code zip archive csv).include? req['type'] errors.add(:upload_requirements, "the type for item #{i + 1} is not valid --> #{req['type']}.") end @@ -846,6 +873,11 @@ def remove_scorm_data() reset_scorm_config() end + # Get the path to the task sheet - using the current abbreviation + def task_sheet_filename + FileHelper.sanitized_filename("#{unit.code}-#{abbreviation}-TaskSheet.pdf") + end + # Get the path to the task sheet - using the current abbreviation def task_sheet(create = true) task_sheet_with_abbreviation(abbreviation, create) diff --git a/app/models/user.rb b/app/models/user.rb index 159b9aab7f..46cd79a192 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -19,6 +19,7 @@ class User < ApplicationRecord include UserTiiModule + before_save :stamp_theme_preference_updated_at, if: :will_save_change_to_theme_preference? after_update :move_files_on_username_change, if: :saved_change_to_username? ### @@ -157,6 +158,7 @@ def token_for_text?(a_token, token_type) has_many :engagements, dependent: :restrict_with_exception, inverse_of: :user has_many :engagement_comments, dependent: :restrict_with_exception, inverse_of: :user has_many :auth_tokens, dependent: :destroy, inverse_of: :user + has_many :consumed_lti_tokens, dependent: :destroy, inverse_of: :user has_many :user_oauth_tokens, dependent: :destroy, inverse_of: :user has_many :user_oauth_states, dependent: :destroy, inverse_of: :user has_one :webcal, dependent: :destroy, inverse_of: :user @@ -164,13 +166,31 @@ def token_for_text?(a_token, token_type) has_many :marking_sessions, dependent: :destroy + # Notifications feature + has_many :notifications, dependent: :destroy, inverse_of: :user + has_many :push_subscriptions, dependent: :destroy, inverse_of: :user + has_one :additional_notification_email, dependent: :destroy, inverse_of: :user + has_many :additional_notification_email_audits, dependent: :destroy, inverse_of: :user + # Model validations/constraints + # How often the unit summary email goes out. 'off' stops it without touching + # feedback notifications, which used to be the only switch it had. + DIGEST_FREQUENCIES = %w[off daily weekly monthly].freeze + validates :first_name, presence: true validates :last_name, presence: true validates :role_id, presence: true validates :username, presence: true, uniqueness: { case_sensitive: false } validates :email, presence: true, uniqueness: { case_sensitive: false }, format: { with: /\A([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})\z/i } validates :student_id, uniqueness: true, allow_nil: true + validates :theme_preference, inclusion: { in: %w[light dark system] }, allow_nil: true + validates :digest_frequency, inclusion: { in: User::DIGEST_FREQUENCIES } + + # True when a run at this cadence should mail this user. A run with no cadence + # is the existing weekly job, which predates the preference. + def wants_digest_on?(cadence) + (cadence.presence || 'weekly').to_s == digest_frequency + end validate :can_change_to_role?, if: :will_save_change_to_role_id? # Queries @@ -406,18 +426,6 @@ def self.permissions } end - def self.default - user = new - institution_email_domain = Doubtfire::Application.config.institution[:email_domain] - user.username = 'username' - user.first_name = 'First' - user.last_name = 'Last' - user.email = "XXXXXXX@#{institution_email_domain}" - user.nickname = 'Nickname' - user.role_id = Role.student_id - user - end - def self.role_for(user) user.role end @@ -431,6 +439,19 @@ def valid_auth_tokens auth_tokens.where("auth_token_expiry > :now", now: Time.zone.now) end + def self.display_name_for(first_name:, last_name:, nickname: nil) + preferred_first_name = nickname.to_s.strip.presence || first_name.to_s.strip + [preferred_first_name, last_name.to_s.strip].reject(&:blank?).join(' ') + end + + def display_name + self.class.display_name_for( + first_name: first_name, + last_name: last_name, + nickname: nickname + ) + end + def name fn = first_name.split.first # fn = nickname @@ -607,4 +628,10 @@ def get_marking_sessions(unit, start_date: nil, end_date: nil, timezone: nil) unit_role.get_marking_sessions(start_date: start_date, end_date: end_date, timezone: timezone) end end + + private + + def stamp_theme_preference_updated_at + self.theme_preference_updated_at = theme_preference.nil? ? nil : Time.current + end end diff --git a/app/services/additional_notification_email_service.rb b/app/services/additional_notification_email_service.rb new file mode 100644 index 0000000000..ddc6fed067 --- /dev/null +++ b/app/services/additional_notification_email_service.rb @@ -0,0 +1,136 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailService + class InvalidToken < StandardError; end + class AlreadyVerified < StandardError; end + class RateLimited < StandardError; end + # Raised in place of a mail delivery error. SMTP rejections often quote the + # recipient, and Sidekiq logs the message and keeps it with the retry in + # Redis, so only the original class name is carried forward. + class DeliveryFailed < RuntimeError; end + + VERIFICATION_LIFETIME = 24.hours + RATE_LIMIT_WINDOW = 1.hour + RATE_LIMIT_COUNT = 3 + + def self.request(user:, email:) + record = nil + version = nil + + user.with_lock do + enforce_rate_limit!(user) + + record = user.additional_notification_email || user.build_additional_notification_email + replacement = record.persisted? && !record.email.to_s.casecmp?(email.to_s.strip) + + prepare_pending_record(record, email) + record.save! + + audit!(user, 'address_replaced') if replacement + audit!(user, 'verification_requested') + version = record.verification_version + end + + AdditionalNotificationEmailVerificationJob.perform_async(record.id, version) + record + end + + def self.resend(user:) + record = nil + version = nil + + user.with_lock do + enforce_rate_limit!(user) + record = user.additional_notification_email + raise ActiveRecord::RecordNotFound if record.nil? + raise AlreadyVerified if record.verified? + + prepare_pending_record(record, record.email) + record.save! + audit!(user, 'verification_resent') + audit!(user, 'verification_requested') + version = record.verification_version + end + + AdditionalNotificationEmailVerificationJob.perform_async(record.id, version) + record + end + + def self.verify(token:) + record = AdditionalNotificationEmail.record_for_token(token) + raise InvalidToken if record.nil? + + # Lock the user first, as request, resend and remove do. Locking the + # address row first and then inserting the audit row (whose foreign key + # takes a shared lock on the user) could deadlock against them. Holding + # the user lock also serialises this with every change to the address. + record.user.with_lock do + # Validate the signed version again under the lock so a concurrent + # replacement/resend cannot make an earlier token verify the newly + # written address between the first lookup and this lock. + current_record = AdditionalNotificationEmail.record_for_token(token) + raise InvalidToken unless current_record&.id == record.id + raise AlreadyVerified if current_record.verified? + raise InvalidToken if current_record.verification_expired? + + current_record.update!(verified_at: Time.current) + audit!(current_record.user, 'verified') + record = current_record + end + + record + end + + def self.remove(user:) + user.with_lock do + record = user.additional_notification_email + return false if record.nil? + + audit!(user, 'removed') + record.destroy! + end + + true + end + + def self.audit!(user, event) + user.additional_notification_email_audits.create!(event: event) + end + + # Delivery has already happened (or failed) by the time these operational + # audit events are written. An audit-store outage must not retry and + # duplicate accepted mail, or mask the original SMTP failure. + def self.audit_delivery_event(user, event) + audit!(user, event) + rescue StandardError => e + Rails.logger.error( + "Additional notification email audit failed for user_id=#{user.id} " \ + "event=#{event}: #{e.class}" + ) + nil + end + + def self.raise_sanitized_delivery_failure!(error) + raise DeliveryFailed, "Mail delivery failed (#{error.class})", cause: nil + end + + def self.prepare_pending_record(record, email) + record.assign_attributes( + email: email, + verified_at: nil, + verification_version: record.verification_version.to_i + 1, + verification_sent_at: Time.current, + verification_expires_at: Time.current + VERIFICATION_LIFETIME + ) + end + private_class_method :prepare_pending_record + + def self.enforce_rate_limit!(user) + count = user.additional_notification_email_audits + .where(event: 'verification_requested') + .where(created_at: RATE_LIMIT_WINDOW.ago..) + .count + raise RateLimited if count >= RATE_LIMIT_COUNT + end + private_class_method :enforce_rate_limit! +end diff --git a/app/services/digest_delivery_guard.rb b/app/services/digest_delivery_guard.rb new file mode 100644 index 0000000000..05bcb0d5f5 --- /dev/null +++ b/app/services/digest_delivery_guard.rb @@ -0,0 +1,77 @@ +# frozen_string_literal: true + +# Stops a student getting two copies of the same summary email. +# +# The digest had no guard of any kind. Project#send_weekly_status_email calls +# the mailer with deliver_now and records nothing afterwards, so every extra run +# of a cadence was another copy in every inbox: a schedule that fires twice, an +# operator running the rake task to see what it does, or a worker that died +# halfway through a sweep and had Sidekiq retry the whole thing. One line in a +# crontab hid that, because one line cannot fire twice. +# +# A claim is one Redis key per student per run period, held a little longer than +# the period so a late repeat still finds it. Redis rather than a table because +# the claim is worthless once its period is over and Sidekiq already needs Redis +# for the job to exist at all. The production redis-sidekiq runs with appendonly +# on, so claims survive a restart of that container. +# +# Claimed immediately before the send and released if the send raises, so a +# delivery that failed is still retryable and one that worked is not repeatable. +class DigestDeliveryGuard + # How long a claim outlives its own period. Long enough for a run that is late + # or repeated, short enough that nothing accumulates. + TTL = { + 'daily' => 2.days, + 'weekly' => 2.weeks, + 'monthly' => 70.days + }.freeze + + DEFAULT_TTL = 2.weeks + + # Name the period a run belongs to. + # + # Local time and not Time.zone, deliberately. The app leaves config.time_zone + # at UTC, but sidekiq-cron reads config/schedule.yml in the process zone, so + # "7am" is 7am in TZ. Naming the period the same way is what keeps a second + # run on the same local day inside the same period. On UTC dates the 7am + # Melbourne run lands at 21:00 the evening before, and a re-run a few hours + # later would fall on the next UTC date and mail everybody again. + # getlocal rather than a bare Time.now only because Rails/TimeZone wants the + # zone said out loud. It is the process zone either way, which is the point. + def self.period_for(cadence, now = Time.now.getlocal) + case cadence.to_s + when 'daily' then "daily:#{now.strftime('%Y-%m-%d')}" + when 'weekly' then "weekly:#{now.strftime('%G-W%V')}" + when 'monthly' then "monthly:#{now.strftime('%Y-%m')}" + end + end + + # True when this run is the one that gets to mail this user. + def self.claim(user, period) + return true if period.blank? + + ttl = TTL.fetch(period.split(':').first, DEFAULT_TTL).to_i + + Sidekiq.redis { |redis| redis.call('SET', key(user, period), '1', 'NX', 'EX', ttl) } == 'OK' + rescue StandardError => e + # A guard that cannot answer says no. If Redis is unreachable then so is the + # only record of who has already been mailed, and a second copy of a summary + # is worse than a summary that is one period late. + Rails.logger.error "Digest guard unavailable for user #{user.id}: #{e.class} - #{e.message}" + false + end + + # Hand the period back after a send that raised. + def self.release(user, period) + return if period.blank? + + Sidekiq.redis { |redis| redis.call('DEL', key(user, period)) } + rescue StandardError => e + Rails.logger.error "Digest guard could not release user #{user.id}: #{e.class} - #{e.message}" + end + + def self.key(user, period) + "doubtfire:digest:#{period}:#{user.id}" + end + private_class_method :key +end diff --git a/app/services/notification_delivery_policy.rb b/app/services/notification_delivery_policy.rb new file mode 100644 index 0000000000..f28769887a --- /dev/null +++ b/app/services/notification_delivery_policy.rb @@ -0,0 +1,35 @@ +# frozen_string_literal: true + +# Shared by all event producers. Limits are deliberately conservative defaults; +# operators can change them through environment configuration after measurement. +class NotificationDeliveryPolicy + def self.positive_integer(name, default) + value = Integer(ENV.fetch(name, default.to_s), 10) + raise ArgumentError, "#{name} must be a positive integer" unless value.positive? + + value + end + + def self.fanout_allowed?(event:, trigger:, recipient_count:, allow_large_fanout: false) + limit = positive_integer('DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT', 500) + return true if recipient_count <= limit + + allowed = allow_large_fanout == true + Rails.logger.warn({ event: 'notifications.fanout_limit', notification_event: event, + trigger: trigger, recipient_count: recipient_count, + limit: limit, allowed: allowed }.to_json) + allowed + end + + # Called while holding the recipient row lock, so concurrent producers share + # one quota. Throttled events remain visible in-app but send neither channel. + def self.throttled?(user) + limit = positive_integer('DOUBTFIRE_NOTIFICATION_RECIPIENT_LIMIT', 30) + window = positive_integer('DOUBTFIRE_NOTIFICATION_RECIPIENT_WINDOW_SECONDS', 3600) + # A locking read sees current rows even inside an older REPEATABLE READ + # transaction snapshot (some producers already hold a project transaction). + candidates = Notification.where(user_id: user.id).where('created_at >= ?', Time.current - window) + .where.not(email_delivery_state: 'throttled').select(:id).limit(limit) + Notification.current_rows(candidates).length >= limit + end +end diff --git a/app/services/notification_service.rb b/app/services/notification_service.rb new file mode 100644 index 0000000000..1ca04a68b1 --- /dev/null +++ b/app/services/notification_service.rb @@ -0,0 +1,196 @@ +# Central entry point for raising a notification. +# +# Creates the in-app record and fans out to the enabled delivery channels +# (email and push through Sidekiq). A single category toggle (the user's +# receive_*_notifications preference) gates every channel: if the category is +# off, the notification is suppressed entirely. A category listed in +# Notification::CHANNEL_PREFERENCES_FOR_TYPE (so far only unit_hub) also has +# its own email and push opt-ins, checked when those channels are queued and +# again when their jobs run. +# +# Usage: +# NotificationService.notify( +# user: project.student, +# type: 'feedback', +# event: 'task_comment_created', +# message: "New feedback is ready for #{task_definition.name}.", +# link: "/projects/#{project.id}" +# ) +class NotificationService + # Raise a notification for a user. Returns the created Notification, or nil if + # the user's preference suppresses this category. + # + # type - the category the user's preference switches on, one of + # Notification::TYPES. + # event - the specific thing that happened, e.g. 'task_comment_created'. + # Required, so every notification can be traced back to its source. + # notifiable - the record the event happened to, e.g. the comment or the task. + # Optional, so an existing caller and a general notification both + # still work. Supplying it is what lets the notification be cleared + # when the user reads the thing it was about. + def self.notify(user:, type:, event:, message:, link: nil, dedupe_key: nil, notifiable: nil) + notification = reserve( + user: user, + type: type, + event: event, + message: message, + link: link, + dedupe_key: dedupe_key, + notifiable: notifiable + ) + + deliver(notification) + end + + # Persist a notification without running its delivery channels. Callers that + # need a short eligibility lock can commit this reservation, release the + # lock, and then call `deliver` without holding a row lock across network I/O. + def self.reserve(user:, type:, event:, message:, link: nil, dedupe_key: nil, notifiable: nil) + type = type.to_s + User.transaction do + User.with_current_row_access do + # An assessment may already hold an older repeatable-read snapshot. + # Lock the current recipient row and recheck its current preference, so a + # concurrent profile update cannot abort that surrounding assessment. + user = User.current_rows(User.where(id: user.id).limit(1)).first + raise ActiveRecord::RecordNotFound, 'Notification recipient no longer exists' if user.nil? + return nil unless deliver_to?(user, type) + + throttled = NotificationDeliveryPolicy.throttled?(user) + create_notification( + user: user, + notification_type: type, + event: event.to_s, + message: message, + link: link, + dedupe_key: dedupe_key, + notifiable: notifiable, + email_delivery_state: throttled ? 'throttled' : 'pending' + ) + end + end + end + + def self.deliver(notification) + return nil if notification.nil? + return notification if notification.delivered_at? || notification.email_delivery_state == 'throttled' + + if notification.reserved_from_existing_event + ActiveRecord.after_all_transactions_commit do + deliver(Notification.find_by(id: notification.id)) + end + return notification + end + + # Concurrent or retried fan-outs can reserve the same immutable event. A + # lock on that notification (not on the student's project) serializes only + # its push hand-off. Email is queued once by Notification's after_commit + # hook, so it cannot be consumed before an enclosing transaction commits + # and a dedupe retry cannot queue it twice. delivered_at tracks the async + # push hand-off; a failed hand-off stays retryable. + notification.with_lock do + unless notification.delivered_at? + push_queued = queue_push(notification) + notification.update!(delivered_at: Time.current) if push_queued + end + end + + notification + end + + # Whether the user's category preference allows this notification type. + # + # channel - nil for the category as a whole (the in-app record), or :email or + # :push. A channel is only asked about for categories listed in + # Notification::CHANNEL_PREFERENCES_FOR_TYPE, and the category itself + # has to be on first. + def self.deliver_to?(user, type, channel: nil) + pref = Notification::PREFERENCE_FOR_TYPE[type.to_s] + return false unless pref.nil? || user.public_send(pref) + + channel.nil? || channel_enabled?(user, type, channel) + end + + # Whether the user opted in to one delivery channel for a category. True for + # categories that have no per-channel columns. + def self.channel_enabled?(user, type, channel) + column = Notification::CHANNEL_PREFERENCES_FOR_TYPE.dig(type.to_s, channel.to_sym) + column.nil? || user.public_send(column) + end + + # A non-null dedupe key is an immutable event identity. The unique database + # index makes concurrent fan-out jobs race safely: exactly one insert wins, + # and only that winner runs the after_commit hook that queues the email. + def self.create_notification(**attributes) + Notification.transaction(requires_new: true) do + Notification.create!(**attributes) + end + rescue ActiveRecord::RecordNotUnique + raise if attributes[:dedupe_key].blank? + + winner = Notification.current_rows(Notification.where( + user: attributes.fetch(:user), dedupe_key: attributes.fetch(:dedupe_key) + ).limit(1)).first + raise ActiveRecord::RecordNotFound, 'Reserved notification no longer exists' if winner.nil? + + winner.reserved_from_existing_event = true + winner + end + private_class_method :create_notification + + # Email channel. Called from Notification's after_commit hook, never directly + # from notify or deliver, so the notification is committed before the job exists. + # + # Queue only the stable Notification id; message content, recipient details + # and other student data remain in the database. Queue connection errors are + # best-effort so the in-app record and push delivery are not blocked. Delivery + # failures are raised by the job for Sidekiq to retry. + def self.queue_email(notification) + return if notification.email_delivery_state == 'throttled' + + # A category with its own email opt-in skips the job entirely when the + # recipient has not opted in, so a cohort-wide announcement does not queue + # hundreds of jobs that would only return early. The job asks again anyway. + # Record the outcome, so the row does not sit in 'pending' for good. + unless channel_enabled?(notification.user, notification.notification_type, :email) + Notification.where(id: notification.id, email_delivery_state: 'pending').update_all( # rubocop:disable Rails/SkipsModelValidations + email_delivery_state: 'suppressed' + ) + return false + end + + if Rails.env.development? && ENV['DOUBTFIRE_NOTIFICATION_EMAIL_INLINE'] == 'true' + NotificationEmailJob.new.perform(notification.id) + else + NotificationEmailJob.perform_async(notification.id) + end + rescue StandardError => e + # A lost Redis reply can race a worker that already accepted the job. + # Preserve its durable outcome rather than overwriting it from this stale object. + Notification.where(id: notification.id, email_delivery_state: 'pending').update_all( # rubocop:disable Rails/SkipsModelValidations + email_delivery_state: 'queue_failed', email_delivery_error_class: e.class.name + ) + Rails.logger.error( + "Failed to queue notification email for Notification #{notification.id}: #{e.class}" + ) + false + end + + # Push channel. Queue only the stable Notification id so no student or + # notification content is copied into Redis. A failed hand-off leaves + # delivered_at unset, allowing the existing availability retry to try the + # push hand-off again without duplicating the after-commit email. + def self.queue_push(notification) + # Nothing to hand off when the recipient has not opted in to push for this + # category. Counts as handed off, so a retry does not try again. + return true unless channel_enabled?(notification.user, notification.notification_type, :push) + + PushNotificationDeliveryJob.perform_async(notification.id) + rescue StandardError => e + Rails.logger.error( + "Failed to queue notification push for Notification #{notification.id}: #{e.class}" + ) + false + end + private_class_method :queue_push +end diff --git a/app/services/push_notification_service.rb b/app/services/push_notification_service.rb new file mode 100644 index 0000000000..e74682a93b --- /dev/null +++ b/app/services/push_notification_service.rb @@ -0,0 +1,264 @@ +# Web Push delivery channel. +# +# PushNotificationDeliveryJob calls this for every notification handed off by +# NotificationService. Two properties make that safe: +# +# * without VAPID keys it is a development no-op; production boot validates +# the matching key pair before any requests or jobs are accepted +# * one browser failing never stops attempts to the others; transient failures +# are raised only after fan-out so Sidekiq can retry the delivery job +# +# Because the shared fan-out queues the job, every event that queues an email +# also queues a push, with no per-event work. +# +# Key generation and setup: docs/notifications/push-setup.md. +class PushNotificationService + class DeliveryError < StandardError; end + + # Push services reject a payload much over 4KB once encrypted. Nothing here + # comes close, but the message is user-facing text assembled from names and + # task titles, so it is trimmed rather than trusted. + MAX_BODY_LENGTH = 400 + + # Lock-screen copy is a separate channel from the richer in-app notification + # and email. These v2 events contain free-form names or precise schedule data + # in Notification#message, so use bounded, reviewed copy whenever the payload + # is rendered. Keeping the decision here means direct delivery and any future + # payload regeneration cannot accidentally bypass the privacy boundary. + LOCK_SCREEN_BODY_OVERRIDES = { + 'tutorial_changed' => 'Your tutorial details changed.', + 'group_membership_changed' => 'Your group membership changed.', + 'task_submitted' => 'A task is ready for marking.', + # Same shape as task_submitted: the raw message names the student and task. + 'task_help_requested' => 'A student asked for help with a task.', + 'portfolio_received' => 'Your portfolio submission was received.', + # The raw message names the outcome and a precise date ("Extension rejected + # for 1.1P, was due Mon Sep 14"), a result and a date both banned on a lock + # screen by the MN-S04 rule. This event predates that review, so it had no + # override and went out raw. Reveal only that an assessment happened. + 'extension_assessed' => 'An extension request was assessed.', + # The raw message names the student and task, like task_submitted. + 'extension_requested' => 'A student asked for an extension.', + # The raw message names the student. + 'portfolio_submitted' => 'A student submitted a portfolio.', + # Unit Hub messages carry the announcement title, or a session's title, + # time and room. Say only what kind of thing happened. + 'unit_announcement_published' => 'There is a new announcement in your unit.', + 'unit_announcement_updated' => 'An announcement in your unit was updated.', + 'unit_session_changed' => 'A session in your unit has changed.', + 'unit_session_starting_soon' => 'A session in your unit starts soon.' + }.freeze + + # MN-C03 BEGIN: safe click route constants + SAFE_CLICK_FALLBACK = '/notifications'.freeze + MAX_CLICK_LINK_LENGTH = 256 + # The percent sign is no longer forbidden outright, because the task segment + # now carries %20 for an abbreviation with a space in it. Everything else a + # percent sign can introduce is still refused by the task route below, which + # admits %20 and no other escape. + FORBIDDEN_CLICK_LINK_TEXT = /[\u0000-\u001f\u007f\s\\?#]/ + SAFE_PROJECT_ROOT_LINK = %r{\A/projects/[1-9]\d*/(?:dashboard|groups)\z} + # %20 only, deliberately not "a percent and any two hex digits". %2F is an + # encoded slash and %5C an encoded backslash, and either walks a path + # separator straight past an anchored pattern written on the assumption there + # is none. The cap is 128 rather than 32 because abbreviations already in + # these repos run past 40 characters; it is not raised further because + # MAX_CLICK_LINK_LENGTH is checked first and would decide anyway. + SAFE_PROJECT_TASK_LINK = %r{\A/projects/[1-9]\d*/dashboard/[A-Za-z0-9](?:[A-Za-z0-9._-]|%20){0,127}(?:/feedback)?\z}x + # The Unit Hub opens on a unit and one announcement or session. This is the + # only destination allowed a query string, and only in exactly this shape of + # two numeric ids, so it is matched whole before the text screen below. + SAFE_UNIT_HUB_LINK = %r{\A/unit-hub\?unit=[1-9]\d{0,9}&(?:announcement|session)=[1-9]\d{0,9}\z} + # MN-C03 END: safe click route constants + # Seconds. web-push sets no timeouts of its own, so without these a push + # service that accepts a connection and then never answers holds a Sidekiq + # worker thread and reduces delivery capacity until the process kills it. + # + # Both are passed together on purpose. web-push 3.0.1 guards read_timeout on + # open_timeout being present (lib/web_push/request.rb line 15), so passing + # read_timeout alone silently does nothing. + OPEN_TIMEOUT = 5 + READ_TIMEOUT = 5 + SSL_TIMEOUT = 5 + + # Push services otherwise retain messages for the gem default of four weeks. + # OnTrack notifications describe current workflow state, so delivering one + # days or weeks later is misleading. One hour tolerates a short disconnect + # without surfacing stale due-date or status alerts. + MESSAGE_TTL = 1.hour.to_i + MESSAGE_URGENCY = 'normal'.freeze + + def self.deliver(notification) + return unless configured? + + subscriptions = notification.user.push_subscriptions.to_a + return if subscriptions.empty? + + payload = payload_for(notification) + + failures = [] + subscriptions.each do |subscription| + deliver_to(subscription, payload) + rescue StandardError => e + # Finish the fan-out before raising. This preserves delivery to healthy + # browsers while ensuring a provider outage reaches Sidekiq's retry path. + Rails.logger.error "Failed to push to subscription #{subscription.id}: #{e.class}" + failures << e + end + + return if failures.empty? + + raise DeliveryError, + "Push delivery failed for #{failures.length} subscription(s)", + cause: failures.first + end + + # The shape Angular's own ngsw-worker.js understands. It looks for a top level + # "notification" key and displays the notification itself, which is why none of + # this needs a hand written service worker. Use any other shape and somebody + # has to write one. + # + # data.link is what MN-C03 reads to decide where to send the user on click. + # + # tag and renotify are both in the service worker's own list of forwarded + # option names (ngsw-worker.js, NOTIFICATION_OPTION_NAMES), so they reach + # showNotification without any change on the web side. + def self.payload_for(notification) + click_link = safe_click_link(notification.link) + + { + notification: { + title: Doubtfire::Application.config.institution[:product_name], + body: body_for(notification), + tag: tag_for(notification), + icon: '/assets/icons/android-chrome-192x192.png', + badge: '/assets/icons/android-chrome-192x192.png', + # False, so a replacement updates the banner without making a sound or + # vibrating again. + # + # A burst is the case this exists for: a tutor working through one task + # posts five comments in two minutes. The tag already collapses those + # into one banner, and renotify: true would put the buzz back on every + # one of them, which is most of what made the burst worth collapsing. + # The user has already been interrupted once and the banner is already + # on their screen saying the newest thing. + # + # The cost is that a genuinely new message inside an ongoing + # conversation arrives silently while the old banner is still up. That + # is the right way round: the person has been told, and the alternative + # is being told five times. + renotify: false, + data: { + notification_id: notification.id, + link: click_link, + onActionClick: { + default: { + operation: 'focusLastFocusedOrOpen', + url: click_link + } + } + } + } + }.to_json + end + + def self.safe_click_link(link) + return SAFE_CLICK_FALLBACK unless link.is_a?(String) + return SAFE_CLICK_FALLBACK if link.empty? || link.length > MAX_CLICK_LINK_LENGTH + return SAFE_CLICK_FALLBACK unless link == link.strip + return link if link.match?(SAFE_UNIT_HUB_LINK) + return SAFE_CLICK_FALLBACK if link.match?(FORBIDDEN_CLICK_LINK_TEXT) + return link if link == SAFE_CLICK_FALLBACK || link.match?(SAFE_PROJECT_ROOT_LINK) + return link if link.match?(SAFE_PROJECT_TASK_LINK) + + SAFE_CLICK_FALLBACK + end + + def self.body_for(notification) + LOCK_SCREEN_BODY_OVERRIDES + .fetch(notification.event.to_s, notification.message.to_s) + .truncate(MAX_BODY_LENGTH) + end + + # Use the event and validated destination as the collapse key so repeated + # pushes about the same event and task can replace one banner. + # + # notification_type is intentionally not used because several different + # events share one category. For example, a task status change must not + # silently replace a due-date warning about the same task. + # + # A missing or rejected destination receives a notification-specific tag. + # This prevents unrelated downgraded notifications from replacing each other + # and prevents the rejected raw route from being copied into the tag. + def self.tag_for(notification) + click_link = safe_click_link(notification.link) + return "notification-#{notification.id}" unless click_link == notification.link + + "#{notification.event}:#{click_link}" + end + + def self.deliver_to(subscription, payload) + # Checked again here rather than trusted from the row. PushSubscription + # validates this on write, but rows created before that validation existed + # were never checked, and this is the line that actually makes the outbound + # request. Refusing here is what stops a stored bad endpoint being used. + unless PushSubscription.push_service_endpoint?(subscription.endpoint) + Rails.logger.error "Refusing to push to subscription #{subscription.id}: endpoint is not a recognised push service" + return + end + + # Checked here for the same reason as the endpoint: a row written before the + # key validation existed can still hold a malformed key. web-push would fail + # to encrypt against it with an error that is neither ExpiredSubscription nor + # InvalidSubscription, so the fan-out below would treat it as a transient + # failure and Sidekiq would retry, re-sending to this user's healthy devices + # every time. Skip the bad row instead. New rows are rejected on write. + unless PushSubscription.valid_web_push_key?(subscription.p256dh, PushSubscription::P256DH_BYTES) && + PushSubscription.valid_web_push_key?(subscription.auth, PushSubscription::AUTH_BYTES) + Rails.logger.error "Refusing to push to subscription #{subscription.id}: keys are not valid web push material" + return + end + + WebPush.payload_send( + message: payload, + endpoint: subscription.endpoint, + p256dh: subscription.p256dh, + auth: subscription.auth, + vapid: vapid_details, + ttl: MESSAGE_TTL, + urgency: MESSAGE_URGENCY, + open_timeout: OPEN_TIMEOUT, + read_timeout: READ_TIMEOUT, + ssl_timeout: SSL_TIMEOUT + ) + rescue WebPush::ExpiredSubscription, WebPush::InvalidSubscription => e + # 410 or 404. The browser has thrown this registration away, or it was never + # valid. Nothing will ever reach it again, so drop the row rather than retry + # it on every notification from here to the end of time. + Rails.logger.info "Removing dead push subscription #{subscription.id}: #{e.class}" + subscription.destroy + end + + def self.vapid_details + { + subject: vapid_subject, + public_key: ENV.fetch('DOUBTFIRE_VAPID_PUBLIC_KEY'), + private_key: ENV.fetch('DOUBTFIRE_VAPID_PRIVATE_KEY') + } + end + + # Push services want a way to contact whoever is sending, as a mailto: or a + # URL. They reject the request outright if it is missing. + def self.vapid_subject + ENV['DOUBTFIRE_VAPID_SUBJECT'].presence || + Doubtfire::Application.config.institution[:host].presence || + 'mailto:noreply@doubtfire.local' + end + + # True once VAPID keys are configured. Keeps the fan-out safe to call today. + def self.configured? + ENV['DOUBTFIRE_VAPID_PUBLIC_KEY'].present? && ENV['DOUBTFIRE_VAPID_PRIVATE_KEY'].present? + end + + private_class_method :body_for, :safe_click_link, :deliver_to, :vapid_details, :vapid_subject +end diff --git a/app/sidekiq/additional_notification_email_delivery_job.rb b/app/sidekiq/additional_notification_email_delivery_job.rb new file mode 100644 index 0000000000..e8dff72844 --- /dev/null +++ b/app/sidekiq/additional_notification_email_delivery_job.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailDeliveryJob + include Sidekiq::Job + + # This optional channel retries independently. Its failure can never cause a + # successfully accepted institutional message to be sent again. + sidekiq_options queue: :mailers, retry: 3 + + def perform(notification_id, additional_notification_email_id, verification_version) + # Either row can be deleted after queueing (the notification by delete-all, + # the address by removal). Neither is a delivery failure, so do not retry + # or record one. + notification = Notification.find_by(id: notification_id) + additional = AdditionalNotificationEmail.find_by(id: additional_notification_email_id) + return if notification.nil? || additional.nil? + + return unless additional.user_id == notification.user_id + return unless additional.verified? + return unless additional.verification_version == verification_version + return if additional.email.casecmp?(notification.user.email) + return unless NotificationService.deliver_to?( + notification.user, + notification.notification_type, + channel: :email + ) + + NotificationsMailer.additional_notification_copy(notification, additional.email).deliver_now + AdditionalNotificationEmailService.audit_delivery_event( + notification.user, + 'notification_copy_delivered' + ) + rescue StandardError => e + user_id = notification&.user_id + if notification + AdditionalNotificationEmailService.audit_delivery_event( + notification.user, + 'notification_copy_failed' + ) + end + Rails.logger.error( + "Additional notification copy failed for user_id=#{user_id || 'unknown'} " \ + "notification_id=#{notification_id}: #{e.class}" + ) + # Retry as before, but without the original message, which for an SMTP + # rejection usually quotes the additional address. + AdditionalNotificationEmailService.raise_sanitized_delivery_failure!(e) + end +end diff --git a/app/sidekiq/additional_notification_email_verification_job.rb b/app/sidekiq/additional_notification_email_verification_job.rb new file mode 100644 index 0000000000..75ef8bde5b --- /dev/null +++ b/app/sidekiq/additional_notification_email_verification_job.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +class AdditionalNotificationEmailVerificationJob + include Sidekiq::Job + + # Keep verification secrets and addresses out of Redis. The worker loads the + # current record and creates its short-lived signed token only while rendering + # the email. A version mismatch makes a queued replacement/resend job stale. + sidekiq_options queue: :mailers, retry: 3 + + def perform(additional_notification_email_id, verification_version) + # A removed address leaves nothing to verify, so do not retry. + record = AdditionalNotificationEmail.find_by(id: additional_notification_email_id) + return if record.nil? + return unless record.pending? + return if record.verification_expired? + return unless record.verification_version == verification_version + + begin + AdditionalNotificationEmailMailer.verification(record).deliver_now + rescue StandardError => e + Rails.logger.error( + "Additional notification email verification delivery failed for user_id=#{record.user_id}: #{e.class}" + ) + AdditionalNotificationEmailService.raise_sanitized_delivery_failure!(e) + end + AdditionalNotificationEmailService.audit_delivery_event( + record.user, + 'verification_email_delivered' + ) + end +end diff --git a/app/sidekiq/execute_communication_set_job.rb b/app/sidekiq/execute_communication_set_job.rb index fa279e1124..14b242e958 100644 --- a/app/sidekiq/execute_communication_set_job.rb +++ b/app/sidekiq/execute_communication_set_job.rb @@ -117,7 +117,8 @@ def execute_change_target_grade_action(action, projects) project_id: project.id, username: project.user&.username, previous_target_grade: previous_target_grade, - target_grade: action.target_grade + target_grade: action.target_grade, + executed_at: Time.current } end end @@ -150,16 +151,39 @@ def execute_email_student_action(action, projects, unit, rule) subject = render_template(action.subject, project, unit, rule, projects.length) body = render_template(action.body, project, unit, rule, projects.length) - CommunicationsMailer.communication_email( - to: formatted_email(recipient), - from: sender, - subject: subject, - body: body, - recipient: recipient, - sender: sender_user_for(unit), - unit: unit, - rule: rule - ).deliver_now + begin + CommunicationsMailer.communication_email( + to: formatted_email(recipient), + from: sender, + subject: subject, + body: body, + recipient: recipient, + sender: sender_user_for(unit), + unit: unit, + rule: rule + ).deliver_now + rescue StandardError => e + # One unroutable address used to take the whole run down. The job then + # retried from the top and re-mailed everybody it had already reached, + # because nothing here records who has been sent to. Record the failure + # against the one recipient and carry on. StandardError and not + # Exception, so an Interrupt or a SIGTERM still stops the job. + logger.error( + "ExecuteCommunicationSetJob delivery failed for project #{project.id} " \ + "<#{recipient.email}>: #{e.class} #{e.message}" + ) + + next { + action_id: action.id, + action_type: action.type, + status: 'failed', + project_id: project.id, + username: recipient.username, + recipient_email: recipient.email, + reason: e.message, + executed_at: Time.current + } + end { action_id: action.id, @@ -167,7 +191,8 @@ def execute_email_student_action(action, projects, unit, rule) status: 'sent', project_id: project.id, username: recipient.username, - recipient_email: recipient.email + recipient_email: recipient.email, + executed_at: Time.current } end end @@ -209,7 +234,8 @@ def execute_email_staff_action(action, projects, unit, rule) project_id: project.id, username: project.user&.username, recipient_email: recipient.email, - recipient_username: recipient.username + recipient_username: recipient.username, + executed_at: Time.current } end end @@ -277,7 +303,8 @@ def execute_task_comment_action(action, projects, unit, rule) username: project.user&.username, task_definition_id: task_definition.id, task_definition_name: task_definition.name, - comment_id: comment.id + comment_id: comment.id, + executed_at: Time.current } end end @@ -294,7 +321,7 @@ def send_action_log_to_convenors(projects, unit, rule, prior_action_results) }] end - recipients = unit.convenors.includes(:user).map(&:user).select { |user| user&.email.present? }.uniq(&:id) + recipients = User.joins(:unit_roles).where(unit_roles: { unit_id: unit.id, role_id: Role.convenor_id }).distinct.where.not(email: [nil, '']).to_a if recipients.empty? return [{ @@ -349,7 +376,7 @@ def staff_recipients_for(project, unit, action) end if action.email_convenors - recipients.concat(unit.convenors.includes(:user).map(&:user)) + recipients.concat(User.joins(:unit_roles).where(unit_roles: { unit_id: unit.id, role_id: Role.convenor_id }).distinct.to_a) end recipients.select { |recipient| recipient&.email.present? }.uniq(&:id) @@ -448,6 +475,8 @@ def build_action_log_csv(rule, projects, action_results) elsif result[:status] == 'commented' task_definition = TaskDefinition.find_by(id: result[:task_definition_id]) "Added comment to #{task_definition_label(task_definition)}" + elsif result[:status] == 'failed' + "Failed to send email to #{result[:recipient_email]}: #{result[:reason]}" elsif result[:recipient_email].present? "Sent email to #{result[:recipient_email]}" else @@ -465,7 +494,7 @@ def build_action_log_csv(rule, projects, action_results) # target_grade_name(result[:previous_target_grade]), # target_grade_name(result[:target_grade]), result[:recipient_email], - Time.current.iso8601 + result[:executed_at]&.iso8601 ] end end diff --git a/app/sidekiq/import_students_lti_job.rb b/app/sidekiq/import_students_lti_job.rb index 18a6d3f9f4..47a5a2cbc0 100644 --- a/app/sidekiq/import_students_lti_job.rb +++ b/app/sidekiq/import_students_lti_job.rb @@ -9,6 +9,7 @@ class ImportStudentsLtiJob include MimeCheckHelpers include CsvHelper include LtiHelper + include FederatedIdentityHelper sidekiq_options lock: :until_executed, lock_args_method: ->(args) { [args.first] }, @@ -43,9 +44,10 @@ def perform(unit_id, members) username: member["email"][/(.*)@/, 1] } - user = User.find_by(login_id: user_id_data[:login_id]) || - User.find_by(username: user_id_data[:username]) || - User.find_by(email: user_id_data[:email]) || + user = user_for_asserted_identity(login_id: user_id_data[:login_id], + email: user_id_data[:email], + derived_username: user_id_data[:username], + source: "Lti import of unit #{unit.id}") || User.create! do |new_user| # Update new user with details from the SAML response Doubtfire::Application.config.institution_settings.update_user_from_lti_response( @@ -79,7 +81,8 @@ def perform(unit_id, members) result[:errors] << { row: member, message: "Failed to create user" } end rescue StandardError => e - result[:errors] << { row: member, message: e } + logger.error e + result[:errors] << { row: member, message: "#{e.class}: #{e.message}" } end store(result: result.to_json) diff --git a/app/sidekiq/new_task_available_notification_job.rb b/app/sidekiq/new_task_available_notification_job.rb new file mode 100644 index 0000000000..d98abae425 --- /dev/null +++ b/app/sidekiq/new_task_available_notification_job.rb @@ -0,0 +1,145 @@ +# frozen_string_literal: true + +class NewTaskAvailableNotificationJob + include Sidekiq::Job + + BATCH_SIZE = 100 + EVENT = 'new_task_available' + TYPE = 'task' + + sidekiq_options lock: :until_executed, + lock_args_method: ->(args) { [args.first] }, + on_conflict: :reject, + retry: 3 + + def self.enqueue(task_definition_id) + perform_async(task_definition_id) + rescue StandardError => e + Rails.logger.error( + "Failed to enqueue new-task notification for TaskDefinition #{task_definition_id}: " \ + "#{e.class} - #{e.message}" + ) + nil + end + + def self.track_and_enqueue(task_definition) + task_definition.enable_new_task_notifications! + enqueue(task_definition.id) + rescue StandardError => e + Rails.logger.error( + "Failed to track new-task notification for TaskDefinition #{task_definition.id}: " \ + "#{e.class} - #{e.message}" + ) + enqueue(task_definition.id) + end + + def self.track_and_enqueue_all(task_definitions) + task_definition_ids = task_definitions.map do |task_definition| + begin + task_definition.enable_new_task_notifications! + rescue StandardError => e + Rails.logger.error( + "Failed to track new-task notification for TaskDefinition #{task_definition.id}: " \ + "#{e.class} - #{e.message}" + ) + end + + task_definition.id + end + + perform_bulk(task_definition_ids.map { |id| [id] }) unless task_definition_ids.empty? + rescue StandardError => e + Rails.logger.error( + "Failed to bulk enqueue new-task notifications: #{e.class} - #{e.message}" + ) + nil + end + + def self.deliver(project, task_definition) + notification = nil + + # Recheck mutable eligibility under a short row lock. The reservation is + # committed before channel jobs are handed off; provider network I/O occurs + # in workers and never holds the project lock. + project.with_lock do + project.reload + unit = project.unit + eligible = unit.active && project.enrolled && project.target_grade.present? && + task_definition.target_grade <= project.target_grade + + if eligible + notification = NotificationService.reserve( + user: project.student, + type: TYPE, + event: EVENT, + message: "A new task is available: #{task_definition.abbreviation} in #{unit.code}.", + link: "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}", + dedupe_key: "#{EVENT}:task-definition:#{task_definition.id}" + ) + end + end + + NotificationService.deliver(notification) + end + + def perform(task_definition_id, allow_large_fanout = false) # rubocop:disable Style/OptionalBooleanParameter + task_definition = TaskDefinition.find_by(id: task_definition_id) + return if task_definition.nil? + + # If the workflow's best-effort marker write failed, the queued job repairs + # it before checking availability. A transient database failure raises and + # lets Sidekiq retry instead of losing a future release permanently. + task_definition.enable_new_task_notifications! if task_definition.new_task_notifications_from.nil? + + unit = task_definition.unit + return unless unit.active + + return unless NotificationDeliveryPolicy.fanout_allowed?( + event: EVENT, trigger: "task-definition:#{task_definition.id}", + recipient_count: unit.projects.where(enrolled: true).count, allow_large_fanout: allow_large_fanout + ) + + failed_project_ids = [] + + unit.projects.where(enrolled: true).includes(:user).find_in_batches(batch_size: BATCH_SIZE) do |projects| + tasks = Task.where( + project_id: projects.map(&:id), + task_definition_id: task_definition.id + ).includes({ project: :unit }, task_definition: :grade_due_dates).index_by(&:project_id) + + projects.each do |project| + notify_project(project, task_definition, tasks[project.id]) + rescue StandardError => e + failed_project_ids << project.id + + Rails.logger.error( + "Failed new-task notification for TaskDefinition #{task_definition.id}, " \ + "Project #{project.id}: #{e.class} - #{e.message}" + ) + end + end + + return if failed_project_ids.empty? + + raise "New-task notifications failed for projects: #{failed_project_ids.join(', ')}" + end + + private + + def notify_project(project, task_definition, task) + return if project.target_grade.nil? + return if task_definition.target_grade > project.target_grade + + # A newly created task is only available when the student's effective + # start date has arrived. Webcal applies flexible, grade-specific and + # student-specific dates without creating a Task row just to notify. + available_on = Webcal.start_date_for_task_definition( + task_definition, + task, + project + ) + return if available_on.to_date > Time.zone.today + + self.class.deliver(project, task_definition) + end +end diff --git a/app/sidekiq/notification_email_job.rb b/app/sidekiq/notification_email_job.rb new file mode 100644 index 0000000000..a9765682b9 --- /dev/null +++ b/app/sidekiq/notification_email_job.rb @@ -0,0 +1,94 @@ +# frozen_string_literal: true + +class NotificationEmailJob + include Sidekiq::Job + + # The queue carries only the stable Notification id. Message content, + # recipient details and other student data remain in the database and are + # loaded by the worker. + # + # Student facing email runs on its own queue so it does not wait behind a + # multi minute PDF build or CSV export on the default queue. A worker has to + # be listening on `mailers` for any of this to be picked up. + sidekiq_options queue: :mailers, retry: 3 + + sidekiq_retry_in do |_count, exception| + # SMTP 5xx is a permanent refusal. Keep the job in Sidekiq's dead set + # immediately; retrying cannot correct an invalid recipient. + :kill if exception.is_a?(Net::SMTPFatalError) || exception.is_a?(Net::SMTPSyntaxError) + end + + sidekiq_retries_exhausted do |job, exception| + Notification.where(id: job['args'].first).where.not(email_delivery_state: %w[delivered suppressed throttled]).update_all( # rubocop:disable Rails/SkipsModelValidations + email_delivery_state: 'failed', email_delivery_error_class: exception.class.name + ) + end + + def perform(notification_id) + # Email is queued after commit. A missing row was deleted while queued and + # does not need another attempt. + notification = Notification.find_by(id: notification_id) + return if notification.nil? + return if %w[delivered suppressed throttled failed].include?(notification.email_delivery_state) + + # The institutional address is always the primary delivery. If this fails, + # raise so Sidekiq retries as before and do not mark an optional copy as a + # substitute for the primary channel. + delivery_error = nil + notification.with_lock do + return if %w[delivered suppressed throttled failed].include?(notification.email_delivery_state) + + # Recheck the preference and terminal state under the same lock, so a + # concurrent opted-out worker cannot overwrite a completed delivery. + unless NotificationsMailer.perform_deliveries && + NotificationService.deliver_to?(notification.user, notification.notification_type, channel: :email) + notification.update!(email_delivery_state: 'suppressed') + return + end + + notification.update!(email_delivery_attempts: notification.email_delivery_attempts + 1) + begin + NotificationsMailer.single_notification(notification).deliver_now + notification.update!(email_delivery_state: 'delivered', email_delivered_at: Time.current, + email_delivery_error_class: nil) + rescue StandardError => e + permanent = e.is_a?(Net::SMTPFatalError) || e.is_a?(Net::SMTPSyntaxError) + notification.update!(email_delivery_state: permanent ? 'failed' : 'retrying', + email_delivery_error_class: e.class.name) + Rails.logger.error({ event: 'notifications.email_failed', notification_id: notification.id, + error_class: e.class.name, permanent: permanent }.to_json) + # Commit the audit state before Sidekiq sees the failure. + delivery_error = e + end + end + raise delivery_error if delivery_error + + begin + # Everything from here is optional, including the lookup itself: a + # database error on it must not make Sidekiq retry the primary message. + additional = notification.user.additional_notification_email + return unless additional&.verified? + return if additional.email.casecmp?(notification.user.email) + + AdditionalNotificationEmailDeliveryJob.perform_async( + notification.id, + additional.id, + additional.verification_version + ) + rescue StandardError => e + # The primary message has already been accepted. An optional destination + # cannot make that delivery retry (and potentially duplicate). The copy + # normally has its own retrying job; this branch is only a lookup or queue + # hand-off failure. Log only class/user/record identifiers: never + # address/content. + AdditionalNotificationEmailService.audit_delivery_event( + notification.user, + 'notification_copy_failed' + ) + Rails.logger.error( + "Additional notification copy queue failed for user_id=#{notification.user_id} " \ + "notification_id=#{notification.id}: #{e.class}" + ) + end + end +end diff --git a/app/sidekiq/push_notification_delivery_job.rb b/app/sidekiq/push_notification_delivery_job.rb new file mode 100644 index 0000000000..f315b354de --- /dev/null +++ b/app/sidekiq/push_notification_delivery_job.rb @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +class PushNotificationDeliveryJob + include Sidekiq::Job + + # Keep provider network I/O off `default`. The development stack cannot + # safely consume that queue because it also contains submission/PDF jobs + # whose supporting services are not present there. + sidekiq_options queue: :notifications, retry: 3 + + # Redis carries only the stable database id. The worker reloads the current + # notification and subscription state immediately before delivery. + def perform(notification_id) + # A producer may enqueue from inside a wider database transaction. Raising + # on a not-yet-visible row makes Sidekiq retry after that transaction commits + # instead of acknowledging and permanently dropping the delivery. + notification = Notification.find(notification_id) + + # The category preference was checked when the notification was raised, but + # a retried job can run hours later. Ask again so a preference the user has + # switched off in the meantime stays off, exactly as NotificationEmailJob + # does for the email channel. Without this a queued or retried push still + # fires after the user has opted the category out. + return unless NotificationService.deliver_to?(notification.user, notification.notification_type, channel: :push) + + PushNotificationService.deliver(notification) + end +end diff --git a/app/sidekiq/send_digest_emails_job.rb b/app/sidekiq/send_digest_emails_job.rb new file mode 100644 index 0000000000..ef44ee1b80 --- /dev/null +++ b/app/sidekiq/send_digest_emails_job.rb @@ -0,0 +1,94 @@ +# frozen_string_literal: true + +# Send the student digest for one cadence. +# +# Nothing scheduled this before. The only recurring summary was one line in +# .ci-setup/crontab, `0 7 * * 1 send_weekly_emails.sh`, which ran the per-unit +# mail and knew nothing about daily or monthly. That line is gone and these +# three cadences replace it, registered in config/schedule.yml. +# +# Sidekiq rather than cron for two reasons. The crontab is only installed by +# lib/shell/pdfgen_entry_point.sh, which runs in the pdfgen container, so a +# deployment without LaTeX has never sent a summary at all and the demo stack +# never will. And CheckUnitSimilarityJob already left that crontab for exactly +# this reason, so following it means one scheduler to keep alive instead of two. +# +# This sends NotificationsMailer#student_digest, one mail per student across all +# their units. The older per-unit NotificationsMailer#weekly_student_summary is +# untouched and still works, but nothing schedules it any more: it is now only +# reachable through `rake mailer:send_status_emails`, run by hand. Both on one +# schedule would mean a student gets the digest and a mail per unit in the same +# morning, which is worse than either on its own. +class SendDigestEmailsJob + include Sidekiq::Job + + # 'off' is a digest_frequency but it is not a cadence. User#wants_digest_on? + # only compares the two strings, so a run with cadence 'off' would mail + # exactly the students who asked for no summary at all. + CADENCES = (User::DIGEST_FREQUENCIES - ['off']).freeze + + BATCH_SIZE = 100 + + # Locked per cadence and not across all three. The three runs mail disjoint + # sets of students, so daily overtaking weekly is fine, but a second daily + # starting while the first is still sweeping is not. + sidekiq_options queue: :mailers, + lock: :until_executed, + lock_args_method: ->(args) { ["send-digest-emails-#{args.first}"] }, + on_conflict: :reject, + retry: 1 + + def perform(cadence = 'weekly') + cadence = cadence.to_s + raise ArgumentError, "cadence must be one of #{CADENCES.join(', ')}" unless CADENCES.include?(cadence) + + # The digest mailer is on ui/email-styling and the digest_frequency column + # is on api/notification-links. Until both are merged a checkout can have + # this schedule without the mailer it calls. Failing here is deliberate: the + # run shows up red in Sidekiq rather than quietly mailing nobody, which is + # how the submission queue managed to sit unread for weeks. + unless NotificationsMailer.respond_to?(:student_digest) + raise 'NotificationsMailer#student_digest is missing. The digest mailer is on ui/email-styling and has not merged into this checkout.' + end + + period = DigestDeliveryGuard.period_for(cadence) + + recipients(cadence).find_each(batch_size: BATCH_SIZE) do |user| + deliver(user, cadence, period) + end + end + + private + + # Who this run mails. + # + # digest_frequency picks the cadence, which is User#wants_digest_on? written + # as a query, and receive_feedback_notifications is still the master switch. + # Both, because the new column defaults to weekly for everyone, so checking + # only the cadence would start mailing students who turned summaries off + # before the preference existed. + # + # Joined to projects so a run renders nothing for staff, for graduates and for + # anyone between trimesters. The mailer returns nil for them, but only after + # building a summary for every unit first. + def recipients(cadence) + User.where(digest_frequency: cadence, receive_feedback_notifications: true) + .where(id: Project.where(enrolled: true).joins(:unit).where(units: { active: true }).select(:user_id)) + end + + # One student. Claimed before the send and handed back if the send raises, so + # a delivery that failed is still retryable and one that worked is not + # repeatable. + def deliver(user, cadence, period) + return unless DigestDeliveryGuard.claim(user, period) + + NotificationsMailer.student_digest(user, cadence).deliver_now + rescue StandardError => e + DigestDeliveryGuard.release(user, period) + + # Logged and swallowed per student, the same as the per-unit send does. One + # student with unrenderable data must not stop the rest of the cohort, and + # a raise here would have Sidekiq retry the whole sweep. + Rails.logger.error "Failed #{cadence} digest for user #{user.id}: #{e.class} - #{e.message}" + end +end diff --git a/app/sidekiq/send_due_soon_reminders_job.rb b/app/sidekiq/send_due_soon_reminders_job.rb new file mode 100644 index 0000000000..4c6e67b067 --- /dev/null +++ b/app/sidekiq/send_due_soon_reminders_job.rb @@ -0,0 +1,175 @@ +# frozen_string_literal: true + +# Remind students about work that is nearly due. +# +# Every other notification in this feature hangs off something a person did: a +# comment was posted, a due date was edited, a status changed. A deadline +# approaching is nobody doing anything, so there is no model to hook and this +# has to be swept for on a schedule. config/schedule.yml runs it. +# +# Recipients come from projects and not from Task rows. OnTrack creates a Task +# row the first time anyone touches the task, so the students who have not +# started have no row, and they are exactly the ones a reminder is for. +# +# Nothing here may call Project#task_for_task_definition, which creates the row +# it cannot find, and nothing may call Project#task_definitions_and_status +# either, because that calls it. This reads project.tasks once per project and +# looks the row up in a hash instead. +class SendDueSoonRemindersJob + include Sidekiq::Job + + BATCH_SIZE = 100 + EVENT = 'task_due_soon' + TYPE = 'task' + + # How far ahead counts as soon, in days. + # + # Three, which is long enough to still do something about it over a weekend + # and short enough that the reminder is about this task rather than about the + # rest of the trimester. Project#top_tasks uses seven for the same idea, and + # seven days of warning on a weekly task is most of the tasks a student has, + # which is a list rather than a reminder. + WINDOW_DAYS = 3 + + # The statuses that mean the student still owes work. + # + # :discuss and :demonstrate are deliberately out. Both mean the student has + # submitted and is waiting on a tutor, so telling them their task is due soon + # is both wrong and the kind of wrong that makes people stop reading + # notifications. Everything past those, complete and fail and the rest, is + # finished with as far as a deadline is concerned. + OUTSTANDING_STATUSES = %i[ + not_started + working_on_it + need_help + fix_and_resubmit + redo + ].freeze + + sidekiq_options lock: :until_executed, + lock_args_method: ->(_args) { ['send-due-soon-reminders'] }, + on_conflict: :reject, + retry: 1 + + def perform(allow_large_fanout = false) # rubocop:disable Style/OptionalBooleanParameter + today = Time.zone.today + horizon = today + WINDOW_DAYS.days + failed_project_ids = [] + + # Units first and then their projects, the same shape as + # NewTaskAvailableNotificationJob, so the unit and its task definitions are + # loaded once per cohort rather than once per student. + Unit.where(active: true).find_each(batch_size: BATCH_SIZE) do |unit| + next unless NotificationDeliveryPolicy.fanout_allowed?( + event: EVENT, trigger: "unit:#{unit.id}", + recipient_count: unit.active_projects.count, allow_large_fanout: allow_large_fanout + ) + remind_unit(unit, today, horizon, failed_project_ids) + end + + return if failed_project_ids.empty? + + # Collected and re-raised at the end rather than swallowed, which is what + # NewTaskAvailableNotificationJob does and for the same reason. Logging and + # carrying on would leave perform successful, Sidekiq would schedule no + # retry, and a student whose task is due today is filtered out as overdue + # tomorrow, so that reminder is gone for good. Re-running the whole sweep is + # safe because of the duplicate guard in notify. + raise "Due-soon reminders failed for projects: #{failed_project_ids.join(', ')}" + end + + private + + def remind_unit(unit, today, horizon, failed_project_ids) + # Read once for the whole cohort. Asking per project is where the query + # count runs away: five hundred students against twenty task definitions is + # five hundred of the same query. + task_definitions = unit.task_definitions.to_a + return if task_definitions.empty? + + unit.active_projects + .where.not(target_grade: nil) + .includes(:user) + .find_each(batch_size: BATCH_SIZE) do |project| + remind_project(project, unit, task_definitions, today, horizon) + rescue StandardError => e + failed_project_ids << project.id + + Rails.logger.error( + "Failed due-soon reminders for Project #{project.id}: #{e.class} - #{e.message}" + ) + end + end + + def remind_project(project, unit, task_definitions, today, horizon) + # One query for this student's rows, then look each one up. The row is only + # read, never created. + tasks = project.tasks.includes(:task_status, :task_definition).index_by(&:task_definition_id) + + task_definitions.each do |task_definition| + next if task_definition.target_grade > project.target_grade + + task = tasks[task_definition.id] + next unless outstanding?(task) + + due = due_date_for(task_definition, task, project) + next if due.nil? + + due = due.to_date + next if due < today || due > horizon + + notify(project, unit, task_definition) + end + end + + # Whether this student still owes work on this task. + # + # No row means nobody has touched it, which is not_started by any other name + # and is the state a reminder is most for. + def outstanding?(task) + return true if task.nil? + + OUTSTANDING_STATUSES.include?(task.status) + end + + # When this task is due for this student. + # + # Webcal already answers exactly this question, for exactly this pair of + # cases, and it is what the calendar feed shows the student. Writing it again + # here would mean two answers to "when is this due" that could disagree. + # + # With a Task row it is Task#local_due_date, which knows about extensions and + # about a unit's flexible dates. Without one it is still not simply the task + # definition's target date: on a unit with flexible dates the grade level + # override applies before any row exists, so a grade 2 student can be due days + # away from the unit's own date without ever having opened the task. + def due_date_for(task_definition, task, project) + Webcal.end_date_for_task_definition(task_definition, task, project) + end + + def notify(project, unit, task_definition) + student = project.student + link = "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task_definition.abbreviation)}" + + # One reminder per student per task, ever. + # + # This job runs again tomorrow and the task is still due soon tomorrow, so + # without this the same student is reminded every morning until the deadline + # passes. The index on (user_id, event) is what makes asking cheap enough to + # do once per candidate task. + return if Notification.exists?( + user_id: student.id, + notification_type: TYPE, + event: EVENT, + link: link + ) + + NotificationService.notify( + user: student, + type: TYPE, + event: EVENT, + message: "#{task_definition.abbreviation} in #{unit.code} is due soon.", + link: link + ) + end +end diff --git a/app/sidekiq/send_new_task_available_notifications_job.rb b/app/sidekiq/send_new_task_available_notifications_job.rb new file mode 100644 index 0000000000..74173e2d63 --- /dev/null +++ b/app/sidekiq/send_new_task_available_notifications_job.rb @@ -0,0 +1,127 @@ +# frozen_string_literal: true + +# Notify students when a future-dated task becomes available. +class SendNewTaskAvailableNotificationsJob + include Sidekiq::Job + + BATCH_SIZE = 100 + CATCH_UP_DAYS = 7 + SETTLE_TIME = 1.hour + ROLLING_WRITER_TOLERANCE = 1.minute + + sidekiq_options lock: :until_executed, + lock_args_method: ->(_args) { ['send-new-task-available-notifications'] }, + on_conflict: :reject, + retry: 3 + + def perform(allow_large_fanout = false) # rubocop:disable Style/OptionalBooleanParameter + today = Time.zone.today + failed_project_ids = [] + + Unit.where(active: true).find_each(batch_size: BATCH_SIZE) do |unit| + next unless NotificationDeliveryPolicy.fanout_allowed?( + event: NewTaskAvailableNotificationJob::EVENT, trigger: "unit:#{unit.id}", + recipient_count: unit.active_projects.count, allow_large_fanout: allow_large_fanout + ) + notify_unit(unit, today, failed_project_ids) + end + + return if failed_project_ids.empty? + + raise "New-task availability notifications failed for projects: #{failed_project_ids.join(', ')}" + end + + private + + def notify_unit(unit, today, failed_project_ids) + task_definitions = candidate_task_definitions(unit, today) + return if task_definitions.empty? + + unit.active_projects + .where.not(target_grade: nil) + .includes(:user) + .find_in_batches(batch_size: BATCH_SIZE) do |projects| + tasks_by_project = Task + .where( + project_id: projects.map(&:id), + task_definition_id: task_definitions.map(&:id) + ) + .includes({ project: :unit }, task_definition: :grade_due_dates) + .group_by(&:project_id) + .transform_values { |tasks| tasks.index_by(&:task_definition_id) } + + projects.each do |project| + notify_project( + project, + task_definitions, + tasks_by_project.fetch(project.id, {}), + today + ) + rescue StandardError => e + failed_project_ids << project.id + Rails.logger.error( + "Failed new-task availability notifications for Project #{project.id}: " \ + "#{e.class} - #{e.message}" + ) + end + end + end + + def candidate_task_definitions(unit, today) + # Definitions written by an older process during a rolling deployment get + # the database-default marker. Give multi-step imports/copies time to finish + # before the sweep can observe them; current workflows enqueue explicitly. + tracked = unit.task_definitions + .where.not(new_task_notifications_from: nil) + .where('created_at <= ?', Time.current - SETTLE_TIME) + window = (today - CATCH_UP_DAYS.days).beginning_of_day..today.end_of_day + + ids = tracked.where(created_at: window).ids + ids.concat(tracked.where(start_date: window).ids) + ids.concat( + TaskDefinitionGradeDueDate.where( + task_definition_id: tracked.select(:id), + start_date: window + ).distinct.pluck(:task_definition_id) + ) + ids.concat( + Task.where( + task_definition_id: tracked.select(:id), + target_start_date: window + ).distinct.pluck(:task_definition_id) + ) + ids.concat( + Task.where(task_definition_id: tracked.select(:id)) + .where('extensions < 0') + .distinct + .pluck(:task_definition_id) + ) + + tracked.where(id: ids.uniq).includes(:grade_due_dates).to_a + end + + def notify_project(project, task_definitions, tasks, today) + task_definitions.each do |task_definition| + next if task_definition.target_grade > project.target_grade + + available_on = Webcal.start_date_for_task_definition( + task_definition, + tasks[task_definition.id], + project + ).to_date + + tracking_from = task_definition.new_task_notifications_from.to_date + recently_created = task_definition.created_at >= + task_definition.new_task_notifications_from - ROLLING_WRITER_TOLERANCE && + task_definition.created_at.to_date >= today - CATCH_UP_DAYS.days + release_in_window = available_on.between?( + [tracking_from, today - CATCH_UP_DAYS.days].max, + today + ) + next unless recently_created || release_in_window + next if available_on > today + + NewTaskAvailableNotificationJob.deliver(project, task_definition) + end + end +end diff --git a/app/sidekiq/task_due_date_changed_notification_job.rb b/app/sidekiq/task_due_date_changed_notification_job.rb new file mode 100644 index 0000000000..9c0ed4fc34 --- /dev/null +++ b/app/sidekiq/task_due_date_changed_notification_job.rb @@ -0,0 +1,88 @@ +# frozen_string_literal: true + +class TaskDueDateChangedNotificationJob + include Sidekiq::Job + + BATCH_SIZE = 100 + EVENT = 'task_due_date_changed' + TYPE = 'task' + + sidekiq_options lock: :until_executed, + lock_args_method: ->(args) { args.first(4) }, + on_conflict: :reject, + retry: 3 + + def perform(task_definition_id, _previous_due_date, new_due_date, change_id = nil, allow_large_fanout = false) # rubocop:disable Style/OptionalBooleanParameter + task_definition = TaskDefinition.find_by(id: task_definition_id) + return if task_definition.nil? + return unless task_definition.unit.active + return unless current_due_date(task_definition) == new_due_date + + # A date can be used more than once. New API jobs carry an occurrence id; + # pre-deployment jobs keep their Sidekiq jid across retries. The timestamp + # fallback supports direct invocations without a Sidekiq job envelope. + occurrence = change_id.presence || jid.presence || + "#{new_due_date}:#{task_definition.updated_at.utc.iso8601(6)}" + + return unless NotificationDeliveryPolicy.fanout_allowed?( + event: EVENT, trigger: "task-definition:#{task_definition.id}", + recipient_count: eligible_projects(task_definition).count, allow_large_fanout: allow_large_fanout + ) + + failed_project_ids = [] + + eligible_projects(task_definition).find_each(batch_size: BATCH_SIZE) do |project| + notify_project(project, task_definition, occurrence) + rescue StandardError => e + failed_project_ids << project.id + + Rails.logger.error( + "Failed due-date notification for TaskDefinition " \ + "#{task_definition.id}, Project #{project.id}: " \ + "#{e.class} - #{e.message}" + ) + end + + return if failed_project_ids.empty? + + # Collected and re-raised so Sidekiq retries, matching + # NewTaskAvailableNotificationJob and SendDueSoonRemindersJob. This event + # fires once off a convenor's edit and is never swept for again, so a + # per-project rescue that only logged left one transient insert failure + # losing that student's notification for good. Re-running the whole sweep is + # safe because notify_project carries a dedupe_key: a student already + # notified for this occurrence is found, not emailed a second time. + raise "Due-date-changed notifications failed for projects: #{failed_project_ids.join(', ')}" + end + + private + + def eligible_projects(task_definition) + task_definition.unit + .active_projects + .where( + 'projects.target_grade >= ?', + task_definition.target_grade + ) + .includes(:user) + end + + def current_due_date(task_definition) + task_definition[:due_date]&.to_date&.iso8601 + end + + def notify_project(project, task_definition, occurrence) + NotificationService.notify( + user: project.student, + type: TYPE, + event: EVENT, + message: "The due date for #{task_definition.abbreviation} " \ + "in #{task_definition.unit.code} has changed.", + link: "/projects/#{project.id}/dashboard/" \ + "#{ERB::Util.url_encode(task_definition.abbreviation)}", + # Retries share an occurrence; a later edit back to the same date does + # not. The unique index scopes this key to each recipient. + dedupe_key: "#{EVENT}:task-definition:#{task_definition.id}:#{occurrence}" + ) + end +end diff --git a/app/views/additional_notification_email_mailer/verification.html.erb b/app/views/additional_notification_email_mailer/verification.html.erb new file mode 100644 index 0000000000..5708e68119 --- /dev/null +++ b/app/views/additional_notification_email_mailer/verification.html.erb @@ -0,0 +1,17 @@ +<% content_for :icon do %>verification<% end %> +<% content_for :preheader do %>Verify the additional email address requested in <%= @product_name %>.<% end %> +<% content_for :heading do %>Verify your additional email<% end %> + +

Hello <%= @user.nickname.presence || @user.first_name %>,

+ +<%= render 'layouts/notification_mail/details', body: "Someone asked #{@product_name} to send a copy of your notifications to this address. Confirm it is yours to switch that on.", category: 'verification', label: 'Confirm this address' %> + +<%= render 'layouts/notification_mail/button', url: @verification_url, label: 'Verify this email address' %> + +<%= render 'layouts/notification_mail/note', body: 'The link expires after 24 hours. No notification copies are sent to this address until verification succeeds.' %> + +<% content_for :footer do %> +

+ If you did not request this, you can ignore this message. +

+<% end %> diff --git a/app/views/additional_notification_email_mailer/verification.text.erb b/app/views/additional_notification_email_mailer/verification.text.erb new file mode 100644 index 0000000000..f7240eeb4c --- /dev/null +++ b/app/views/additional_notification_email_mailer/verification.text.erb @@ -0,0 +1,10 @@ +Hello <%= @user.nickname.presence || @user.first_name %>, + +CONFIRM THIS ADDRESS +Someone asked <%= @product_name %> to send a copy of your notifications to this address. Confirm it is yours to switch that on. + +<%= @verification_url %> + +The link expires after 24 hours. No notification copies are sent to this address until verification succeeds. + +If you did not request this, you can ignore this message. diff --git a/app/views/communications_mailer/action_log_email.html.erb b/app/views/communications_mailer/action_log_email.html.erb index 7df5658bf1..265c9dcbff 100644 --- a/app/views/communications_mailer/action_log_email.html.erb +++ b/app/views/communications_mailer/action_log_email.html.erb @@ -1,69 +1,23 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

+<% content_for :icon do %>summary<% end %> +<% content_for :preheader do %><%= @affected_students_count %> students matched this rule.<% end %> +<% content_for :heading do %>Communication rule ran<% end %> +<% if @unit&.name.present? %><% content_for :subheading do %><%= @unit.name %><% end %><% end %> -

Hi <%= @recipient&.nickname.presence || @recipient&.first_name || @recipient&.name %>,

+

Hi <%= @recipient&.nickname.presence || @recipient&.first_name || @recipient&.name %>,

- <% @body_paragraphs.each do |paragraph| %> -

<%= paragraph %>

- <% end %> +<% affected_summary = capture do %> + <%= @affected_students_count %> + student<%= "s" unless @affected_students_count == 1 %> matched this rule and were affected by the configured actions. +<% end %> +<%= render 'layouts/notification_mail/details', body: affected_summary, category: 'summary', label: 'Result' %> -
+<% @body_paragraphs.each do |paragraph| %> +

<%= paragraph %>

+<% end %> -

- <%= @affected_students_count %> - students matched this rule and were affected by the configured actions. -

+<%= render 'layouts/notification_mail/note', body: 'A full log of actions is attached as a CSV.' %> -

A full log of actions has been attached as a CSV.

- -

- Cheers,
- The <%= @doubtfire_product_name %> Team -

- -
- - +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team +

diff --git a/app/views/communications_mailer/action_log_email.text.erb b/app/views/communications_mailer/action_log_email.text.erb index f980a817f3..e6956669f8 100644 --- a/app/views/communications_mailer/action_log_email.text.erb +++ b/app/views/communications_mailer/action_log_email.text.erb @@ -1,14 +1,13 @@ Hi <%= @recipient&.nickname.presence || @recipient&.first_name || @recipient&.name %>, +RESULT +<%= @affected_students_count %> student<%= "s" unless @affected_students_count == 1 %> matched this rule and were affected by the configured actions. + <% @body_paragraphs.each do |paragraph| %> <%= paragraph %> <% end %> ---- - -<%= @affected_students_count %> students matched this rule and were affected by the configured actions. - -A full log of actions has been attached as a CSV. +A full log of actions is attached as a CSV. Cheers, The <%= @doubtfire_product_name %> Team diff --git a/app/views/communications_mailer/communication_email.html.erb b/app/views/communications_mailer/communication_email.html.erb index 7933f82475..faed43ff87 100644 --- a/app/views/communications_mailer/communication_email.html.erb +++ b/app/views/communications_mailer/communication_email.html.erb @@ -1,53 +1,28 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

+<% content_for :icon do %>announcement<% end %> +<% content_for :preheader do %><%= @body_paragraphs.first.to_s.truncate(90) %><% end %> +<% content_for :heading do %>A message from <%= @sender&.name.presence || @doubtfire_product_name %><% end %> +<% if @unit&.name.present? %><% content_for :subheading do %><%= @unit.name %><% end %><% end %> - <% @body_paragraphs.each do |paragraph| %> -

<%= paragraph %>

- <% end %> +

Hi <%= @recipient&.nickname.presence || @recipient&.first_name %>,

-
- - +<%# The convenor wrote this body. Lead with the first paragraph in the callout + so the point of the message survives a glance, and leave the rest as prose + because it is theirs to shape, not this template's. %> +<% if @body_paragraphs.any? %> + <%= render 'layouts/notification_mail/details', body: @body_paragraphs.first, category: 'announcement' %> +<% end %> + +<% @body_paragraphs.drop(1).each do |paragraph| %> +

<%= paragraph %>

+<% end %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @sender&.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/communications_mailer/communication_email.text.erb b/app/views/communications_mailer/communication_email.text.erb index 5ad89c15fb..0b035e6c74 100644 --- a/app/views/communications_mailer/communication_email.text.erb +++ b/app/views/communications_mailer/communication_email.text.erb @@ -1,11 +1,11 @@ -<%# Hi <%= @recipient.nickname.presence || @recipient.first_name %> %> +Hi <%= @recipient&.nickname.presence || @recipient&.first_name %>, <% @body_paragraphs.each do |paragraph| %> <%= paragraph %> <% end %> -<%# Cheers, -The <%= @doubtfire_product_name %> Team on behalf of <%= @sender.name %> %> +Cheers, +The <%= @doubtfire_product_name %> Team on behalf of <%= @sender&.name %> --- diff --git a/app/views/layouts/notification_mail.html.erb b/app/views/layouts/notification_mail.html.erb new file mode 100644 index 0000000000..48a432ad28 --- /dev/null +++ b/app/views/layouts/notification_mail.html.erb @@ -0,0 +1,105 @@ + + + + + + + <%# One palette, always light. A dark variant only reaches the few clients that + honour a style block at all: Gmail and Outlook.com drop or rewrite it, and + several force their own inversion on top, which turned dark text on a dark + card. Declaring light only is also what stops Apple Mail auto-darkening a + template it would otherwise treat as dark-aware. %> + + + <%= message.subject %> + + + +<% + product_name = @doubtfire_product_name.presence || @product_name.presence || 'OnTrack' + # A template names its category once and the badge, the rule above the card + # and anything else tinted all read the same entry. A template that names + # nothing still renders, in the neutral brand colour. + icon_category = content_for?(:icon) ? yield(:icon).to_s.strip : '' + accent = notification_mail_accent(icon_category) +%> + + <% if content_for?(:preheader) %> +
+ <%= yield :preheader %>​ ​ ​ ​ ​ ​  +
+ <% end %> + + + + + + + diff --git a/app/views/layouts/notification_mail.text.erb b/app/views/layouts/notification_mail.text.erb new file mode 100644 index 0000000000..37f0bddbd7 --- /dev/null +++ b/app/views/layouts/notification_mail.text.erb @@ -0,0 +1 @@ +<%= yield %> diff --git a/app/views/layouts/notification_mail/_bar_chart.html.erb b/app/views/layouts/notification_mail/_bar_chart.html.erb new file mode 100644 index 0000000000..b13d6e912e --- /dev/null +++ b/app/views/layouts/notification_mail/_bar_chart.html.erb @@ -0,0 +1,32 @@ +<%# A horizontal bar chart made of nested tables. Each bar is a table cell with a + percentage width and a bgcolor, which is the only way to draw a filled shape + that Outlook, Gmail and Apple Mail all render: a background image is ignored, + an SVG is dropped and an is blocked before it loads. + rows: [label, value, accent] triples. scale: what a full bar is worth, so + several charts in one message can share an axis. + The value is printed as "10 of 16" rather than "10", so the bar is decoration + and never the only place the proportion exists. That covers a screen reader, + a phone where the bars are hidden, and Word where they may not draw. %> +<% bars = rows.reject { |_label, value, _accent| value.to_i.negative? } %> +<% top = [scale.to_i, 1].max %> + + <% bars.each_with_index do |(label, value, accent), index| %> + <% pct = [(value.to_i * 100.0 / top).round, 100].min %> + + + + + + <% end %> +
<%= label %> + + + <% if pct.positive? %> + + <% end %> + <% if pct < 100 %> + + <% end %> + +
  
+ 		<%= value %> of <%= top %>
diff --git a/app/views/layouts/notification_mail/_button.html.erb b/app/views/layouts/notification_mail/_button.html.erb new file mode 100644 index 0000000000..a5efed999c --- /dev/null +++ b/app/views/layouts/notification_mail/_button.html.erb @@ -0,0 +1,11 @@ +<%# The padding lives on the td, not on the anchor. Word drops padding from an + inline-block anchor, which left the label touching the edges of the fill in + Outlook; on the cell it is honoured. The anchor still fills the cell, so the + whole coloured area stays clickable everywhere else. %> + + + + +
+ <%= label %> +
diff --git a/app/views/layouts/notification_mail/_code_chip.html.erb b/app/views/layouts/notification_mail/_code_chip.html.erb new file mode 100644 index 0000000000..05e557a2db --- /dev/null +++ b/app/views/layouts/notification_mail/_code_chip.html.erb @@ -0,0 +1,8 @@ +<%# The task code in a bordered pill. A one-cell table rather than a span with + display:inline-block, because Word drops padding from an inline-block and + left the code touching its own border in Outlook. %> + + + + +
<%= code %>
diff --git a/app/views/layouts/notification_mail/_details.html.erb b/app/views/layouts/notification_mail/_details.html.erb new file mode 100644 index 0000000000..94e9e99269 --- /dev/null +++ b/app/views/layouts/notification_mail/_details.html.erb @@ -0,0 +1,18 @@ +<%# The one fact the message is about, in a tinted block with a rule down the + left. It is the first thing the eye lands on after the heading, so the + recipient can stop reading there and still have the point. + An optional category tints it to match the badge, and an optional label + names what the block is. Both default to the neutral brand styling, so the + existing calls that pass only a body keep rendering as they did. %> +<% callout = notification_mail_style(local_assigns.fetch(:category, '')) %> +<% callout_label = local_assigns[:label] %> + + + + +
+ <% if callout_label.present? %> +

<%= callout_label %>

+ <% end %> + <%= body %> +
diff --git a/app/views/layouts/notification_mail/_facts.html.erb b/app/views/layouts/notification_mail/_facts.html.erb new file mode 100644 index 0000000000..365eac4dae --- /dev/null +++ b/app/views/layouts/notification_mail/_facts.html.erb @@ -0,0 +1,16 @@ +<%# Short labelled rows instead of a paragraph. Takes an array of [label, value] + pairs and drops any pair with no value, so a caller can list everything it + might have and let the partial decide what appears. + Two cells in a table row, not a definition list, because Outlook lays out + with the Word engine and a table is the only thing it measures reliably. %> +<% pairs = rows.reject { |_label, value| value.blank? } %> +<% if pairs.any? %> + + <% pairs.each_with_index do |(label, value), index| %> + + + + + <% end %> +
<%= label %><%= value %>
+<% end %> diff --git a/app/views/layouts/notification_mail/_icon.html.erb b/app/views/layouts/notification_mail/_icon.html.erb new file mode 100644 index 0000000000..2854cf9862 --- /dev/null +++ b/app/views/layouts/notification_mail/_icon.html.erb @@ -0,0 +1,13 @@ +<%# A coloured round cell carrying a single text glyph. + No , so nothing here waits on a client that blocks remote images by + default, and no SVG or icon font, which Outlook and Gmail either drop or + turn into tofu. Outlook ignores border-radius and draws a square, which + still reads as a deliberate badge. + The glyph is decorative. The heading beside it always carries the meaning, + so the message reads in full with images off and with the glyph missing. %> +<% icon = notification_mail_style(category) %> + + + + +
diff --git a/app/views/layouts/notification_mail/_note.html.erb b/app/views/layouts/notification_mail/_note.html.erb new file mode 100644 index 0000000000..f56b5bdebf --- /dev/null +++ b/app/views/layouts/notification_mail/_note.html.erb @@ -0,0 +1,5 @@ +<%# Small print under the action. The caveats these messages carry, that the + comment or the new date is not in the email, matter but they are not the + point, so they sit below the button at a smaller size instead of taking a + full paragraph above it. %> +

<%= body %>

diff --git a/app/views/layouts/notification_mail/_progress.html.erb b/app/views/layouts/notification_mail/_progress.html.erb new file mode 100644 index 0000000000..59c813ee56 --- /dev/null +++ b/app/views/layouts/notification_mail/_progress.html.erb @@ -0,0 +1,16 @@ +<%# A progress bar as two table cells with percentage widths and a bgcolor on + each. No background image and no border trick, so Outlook draws it too. + A bar at nought or full renders as a single cell, because a cell at zero + per cent is a cell some clients will still give a pixel or two to. %> +<% filled = [[value.to_i, 0].max, total.to_i].min %> +<% pct = total.to_i.positive? ? (filled * 100.0 / total.to_i).round : 0 %> + + + <% if pct.positive? %> + + <% end %> + <% if pct < 100 %> + + <% end %> + +
  
diff --git a/app/views/layouts/notification_mail/_stats.html.erb b/app/views/layouts/notification_mail/_stats.html.erb new file mode 100644 index 0000000000..67c4c1b0bc --- /dev/null +++ b/app/views/layouts/notification_mail/_stats.html.erb @@ -0,0 +1,13 @@ +<%# Rows of one number and what it means, the figure first, so a recipient can + find the one they care about without reading all of them. + Takes [value, label] pairs, or [value, label, accent] where the figure is + worth colouring: a count sitting with a tutor gets the ready-for-feedback + blue it has on the task itself. %> + + <% rows.each_with_index do |(value, label, accent), index| %> + + + + + <% end %> +
<%= value %><%= label %>
diff --git a/app/views/layouts/notification_mail/_task_rows.html.erb b/app/views/layouts/notification_mail/_task_rows.html.erb new file mode 100644 index 0000000000..fcaadd5763 --- /dev/null +++ b/app/views/layouts/notification_mail/_task_rows.html.erb @@ -0,0 +1,23 @@ +<%# A scannable list of tasks. Four messages used to render the same bulleted + list with the abbreviation bolded inline, which made the task code and the + task name run together. Here the code sits in its own column so a recipient + can find the one they care about without reading the names. + note_for is optional. Pass a proc to put a short trailing note on a row. %> +<% note_for = local_assigns[:note_for] %> + + <% tasks.each_with_index do |task, index| %> + <% task_url = "#{host}/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(task.task_definition.abbreviation)}" %> + <% note = note_for.nil? ? nil : note_for.call(task) %> + + + + + <% end %> +
+ <%= render 'layouts/notification_mail/code_chip', code: task.task_definition.abbreviation %> + + <%= task.task_definition.name %> + <% if note.present? %> + <%= note %> + <% end %> +
diff --git a/app/views/notifications_mailer/discussion_request_created.html.erb b/app/views/notifications_mailer/discussion_request_created.html.erb new file mode 100644 index 0000000000..286b522655 --- /dev/null +++ b/app/views/notifications_mailer/discussion_request_created.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>discuss<% end %> +<% content_for :preheader do %>A discussion prompt is ready for you in <%= @doubtfire_product_name %>.<% end %> +<% content_for :heading do %>Discussion prompt ready<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'discuss', label: 'Discussion prompt' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "The prompt is not included in this email. Open the task to listen and respond." %> +<% content_for :footer do %> +

+ You are receiving this because your feedback notifications are turned on. + You can change that at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/discussion_request_created.text.erb b/app/views/notifications_mailer/discussion_request_created.text.erb new file mode 100644 index 0000000000..fb36763150 --- /dev/null +++ b/app/views/notifications_mailer/discussion_request_created.text.erb @@ -0,0 +1,13 @@ +Hi <%= @user.name %>, + +DISCUSSION PROMPT +<%= @notification.message %> +<% if @notification.link.present? -%> + +Open the task: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +The prompt is not included in this email. Open the task to listen and respond. + +You are receiving this because your feedback notifications are turned on. +You can turn these emails off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/extension_assessed.html.erb b/app/views/notifications_mailer/extension_assessed.html.erb new file mode 100644 index 0000000000..9767f24cf0 --- /dev/null +++ b/app/views/notifications_mailer/extension_assessed.html.erb @@ -0,0 +1,15 @@ +<% content_for :icon do %>extension<% end %> +<% content_for :preheader do %>Your extension request has been assessed.<% end %> +<% content_for :heading do %>Extension request assessed<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'extension', label: 'Outcome' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "Open the task in #{@doubtfire_product_name} to review the current due date and what is left to do." %> +<% content_for :footer do %> +

+ This notification is sent when an extension request is assessed. +

+<% end %> diff --git a/app/views/notifications_mailer/extension_assessed.text.erb b/app/views/notifications_mailer/extension_assessed.text.erb new file mode 100644 index 0000000000..0ba806eb10 --- /dev/null +++ b/app/views/notifications_mailer/extension_assessed.text.erb @@ -0,0 +1,14 @@ +Hi <%= @user.name %>, + +Your extension request has been assessed. + +OUTCOME +<%= @notification.message %> +<% if @notification.link.present? -%> + +Open the task: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +Open the task in <%= @doubtfire_product_name %> to review the current due date and what is left to do. + +This notification is sent when an extension request is assessed. diff --git a/app/views/notifications_mailer/extension_requested.html.erb b/app/views/notifications_mailer/extension_requested.html.erb new file mode 100644 index 0000000000..ce756c5071 --- /dev/null +++ b/app/views/notifications_mailer/extension_requested.html.erb @@ -0,0 +1,21 @@ +<% content_for :icon do %>extension<% end %> +<% content_for :preheader do %><%= @notification.message.to_s.truncate(90) %><% end %> +<% content_for :heading do %>Extension request<% end %> + +

Hi <%= @user.first_name %>,

+

A student asked for an extension that needs someone to assess it.

+<%= render 'layouts/notification_mail/details', body: @notification.message %> + +

+ Open the task in <%= @doubtfire_product_name %> to read the request and + grant or reject it. The reason the student gave is not included in this email. +

+<% if @notification_url.present? %> + <%= render 'layouts/notification_mail/button', url: @notification_url, label: 'Open the task' %> +<% end %> +<% content_for :footer do %> +

+ You can manage your notification preferences in + your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/extension_requested.text.erb b/app/views/notifications_mailer/extension_requested.text.erb new file mode 100644 index 0000000000..605a1729fa --- /dev/null +++ b/app/views/notifications_mailer/extension_requested.text.erb @@ -0,0 +1,15 @@ +Hi <%= @user.first_name %>, + +<%= @notification.message %> + +Open the task in <%= @doubtfire_product_name %> to read the request and grant or reject it. +The reason the student gave is not included in this email. + +<% if @notification_url.present? -%> +Open the task: <%= @notification_url %> +<% end -%> + +This notification is sent when a student asks for an extension that needs someone to assess it. + +You can manage your notification preferences here: +<%= @unsubscribe_url %> diff --git a/app/views/notifications_mailer/group_membership_changed.html.erb b/app/views/notifications_mailer/group_membership_changed.html.erb new file mode 100644 index 0000000000..9afba0591e --- /dev/null +++ b/app/views/notifications_mailer/group_membership_changed.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>group<% end %> +<% content_for :preheader do %>Your group membership changed in <%= @doubtfire_product_name %>.<% end %> +<% content_for :heading do %>Group membership changed<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'group', label: 'What changed' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: "Open your group in #{@doubtfire_product_name}" %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "You can view your current group information in #{@doubtfire_product_name}." %> +<% content_for :footer do %> +

+ This notification was sent because your group membership changed in + <%= @doubtfire_product_name %>. +

+<% end %> diff --git a/app/views/notifications_mailer/group_membership_changed.text.erb b/app/views/notifications_mailer/group_membership_changed.text.erb new file mode 100644 index 0000000000..ba8bf758d1 --- /dev/null +++ b/app/views/notifications_mailer/group_membership_changed.text.erb @@ -0,0 +1,12 @@ +Hi <%= @user.name %>, + +WHAT CHANGED +<%= @notification.message %> + +This notification was sent because your group membership changed in <%= @doubtfire_product_name %>. + +You can view your current group information in <%= @doubtfire_product_name %>. +<% if @notification.link.present? -%> + +Open your group in <%= @doubtfire_product_name %>: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> diff --git a/app/views/notifications_mailer/new_task_available.html.erb b/app/views/notifications_mailer/new_task_available.html.erb new file mode 100644 index 0000000000..46ebceff67 --- /dev/null +++ b/app/views/notifications_mailer/new_task_available.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>new<% end %> +<% content_for :preheader do %>A new task is now available in <%= @doubtfire_product_name %>.<% end %> +<% content_for :heading do %>New task available<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'new', label: 'New task' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "Open the task to review its details and when it is due." %> +<% content_for :footer do %> +

+ You are receiving this because your task notifications are turned on. + You can change that at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/new_task_available.text.erb b/app/views/notifications_mailer/new_task_available.text.erb new file mode 100644 index 0000000000..244be73319 --- /dev/null +++ b/app/views/notifications_mailer/new_task_available.text.erb @@ -0,0 +1,13 @@ +Hi <%= @user.name %>, + +NEW TASK +<%= @notification.message %> +<% if @notification.link.present? -%> + +Open the task: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +Open the task to review its details and when it is due. + +You are receiving this because your task notifications are turned on. +You can turn these emails off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/portfolio_received.html.erb b/app/views/notifications_mailer/portfolio_received.html.erb new file mode 100644 index 0000000000..a3d66bed01 --- /dev/null +++ b/app/views/notifications_mailer/portfolio_received.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>submission<% end %> +<% content_for :preheader do %>This receipt confirms when the submission was received.<% end %> +<% content_for :heading do %>Submission received<% end %> + +

Hi <%= @user.first_name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'submission', label: 'Receipt' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: "View the current status in #{@doubtfire_product_name}" %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "This receipt confirms when the submission was received. It does not confirm an assessment outcome." %> +<% content_for :footer do %> +

+ You can manage your notification preferences in + your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/portfolio_received.text.erb b/app/views/notifications_mailer/portfolio_received.text.erb new file mode 100644 index 0000000000..262e07fd36 --- /dev/null +++ b/app/views/notifications_mailer/portfolio_received.text.erb @@ -0,0 +1,15 @@ +Hi <%= @user.first_name %>, + +RECEIPT +<%= @notification.message %> + +Open <%= @doubtfire_product_name %> to view its current status. +<% if @notification.link.present? -%> + +View the current status: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +This receipt confirms when the submission was received. It does not confirm an assessment outcome. + +You can manage your notification preferences here: +<%= @unsubscribe_url %> diff --git a/app/views/notifications_mailer/portfolio_submitted.html.erb b/app/views/notifications_mailer/portfolio_submitted.html.erb new file mode 100644 index 0000000000..c558e000b2 --- /dev/null +++ b/app/views/notifications_mailer/portfolio_submitted.html.erb @@ -0,0 +1,18 @@ +<% content_for :icon do %>portfolio<% end %> +<% content_for :preheader do %><%= @notification.message.to_s.truncate(90) %><% end %> +<% content_for :heading do %>Portfolio submitted<% end %> + +

Hi <%= @user.first_name %>,

+

A student you teach has submitted their portfolio.

+<%= render 'layouts/notification_mail/details', body: @notification.message %> + +

The portfolio itself is not attached to this email.

+<% if @notification_url.present? %> + <%= render 'layouts/notification_mail/button', url: @notification_url, label: 'Open the portfolio' %> +<% end %> +<% content_for :footer do %> +

+ You can manage your notification preferences in + your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/portfolio_submitted.text.erb b/app/views/notifications_mailer/portfolio_submitted.text.erb new file mode 100644 index 0000000000..a9727d6494 --- /dev/null +++ b/app/views/notifications_mailer/portfolio_submitted.text.erb @@ -0,0 +1,12 @@ +Hi <%= @user.first_name %>, + +<%= @notification.message %> + +The portfolio itself is not attached to this email. +<% if @notification_url.present? -%> + +Open the portfolio: <%= @notification_url %> +<% end -%> + +You can manage your notification preferences here: +<%= @unsubscribe_url %> diff --git a/app/views/notifications_mailer/single_notification.html.erb b/app/views/notifications_mailer/single_notification.html.erb new file mode 100644 index 0000000000..1029633ab2 --- /dev/null +++ b/app/views/notifications_mailer/single_notification.html.erb @@ -0,0 +1,15 @@ +<% content_for :icon do %>new<% end %> +<% content_for :preheader do %><%= @notification.message.to_s.truncate(90) %><% end %> +<% content_for :heading do %>You have a new notification<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'new' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: "View in #{@doubtfire_product_name}" %> +<% end %> +<% content_for :footer do %> +

+ You are receiving this because your notification preferences allow it. + You can update them at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/single_notification.text.erb b/app/views/notifications_mailer/single_notification.text.erb new file mode 100644 index 0000000000..af0c03b196 --- /dev/null +++ b/app/views/notifications_mailer/single_notification.text.erb @@ -0,0 +1,9 @@ +Hi <%= @user.name %>, + +<%= @notification.message %> +<% if @notification.link.present? -%> + +View it here: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +You can update your notification preferences at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/student_digest.html.erb b/app/views/notifications_mailer/student_digest.html.erb new file mode 100644 index 0000000000..01e0102eb4 --- /dev/null +++ b/app/views/notifications_mailer/student_digest.html.erb @@ -0,0 +1,185 @@ +<% + section_heading = 'margin:0 0 12px 0; font-size:14px; line-height:1.4; font-weight:600; letter-spacing:0.4px; text-transform:uppercase;' + unit_heading = 'margin:28px 0 12px 0; font-size:17px; line-height:1.35; font-weight:700; color:#111827;' + + # The absolute date is always printed next to the relative one. A phrase on its + # own is only true on the day it was rendered. + target_phrase = lambda do |date| + next nil if date.blank? + + days = (date.to_date - Time.zone.today).to_i + when_text = + if days.negative? + "#{days.abs} day#{'s' unless days.abs == 1} past" + elsif days.zero? + 'today' + elsif days == 1 + 'tomorrow' + else + "in #{days} days" + end + "#{date.to_date.strftime('%a %-d %b')}, #{when_text}" + end + + lead_category = + if @show_trend + @completed_in_window.positive? ? 'submission' : 'summary' + elsif @overdue_total.positive? + 'overdue' + elsif @next_up.present? + 'due' + else + 'submission' + end +%> +<% content_for :icon do %><%= lead_category %><% end %> +<% content_for :heading do %>Your <%= @cadence %> digest<% end %> +<% content_for :subheading do %><%= @units.count %> unit<%= "s" unless @units.count == 1 %><% if @show_deadlines && @overdue_total.positive? %> · <%= @overdue_total %> behind target<% end %><% end %> +<% content_for :preheader do %><%= @show_trend ? "#{@completed_in_window} task#{'s' unless @completed_in_window == 1} finished #{@window[:since]}." : (@overdue_total.positive? ? "#{@overdue_total} task#{'s' unless @overdue_total == 1} behind target." : 'Here is what is next across your units.') %><% end %> + +

Hi <%= @student.first_name %>,

+ +<% if @show_trend %> + <%# A monthly email must not lead on a deadline. It leads on the trend, and on + whether that trend gets the student to the end of the unit. %> + <% trend = capture do %> +

<%= @completed_in_window %> + task<%= "s" unless @completed_in_window == 1 %> finished <%= @window[:since] %>, across <%= @units.count %> unit<%= "s" unless @units.count == 1 %>.

+ <% if @remaining_total.positive? %> +

+ <%= @remaining_total %> still to go. + <% if @projected_shortfall.present? && @projected_shortfall.positive? %> + At that rate you finish about <%= @projected_shortfall %> short before your earliest unit ends. + <% elsif @projected_finish.present? %> + At that rate you finish with room to spare. + <% end %> +

+ <% else %> +

Nothing is left outstanding across your units.

+ <% end %> + <% end %> + <%= render 'layouts/notification_mail/details', body: trend, category: lead_category, label: "Your month" %> +<% elsif @overdue_total.positive? %> + <% oldest = @next_up.first %> + <% lead = capture do %> +

<%= @overdue_total %> + task<%= "s" unless @overdue_total == 1 %> across your units <%= @overdue_total == 1 ? 'is' : 'are' %> past <%= @overdue_total == 1 ? 'its' : 'their' %> target date.

+ <% if oldest.present? %> +

Start with <%= oldest[:unit].code %> <%= oldest[:task_definition].abbreviation %>, targeted <%= target_phrase.call(oldest[:target_date]) %>.

+ <% end %> + <% end %> + <%= render 'layouts/notification_mail/details', body: lead, category: 'overdue', label: 'Start here' %> +<% elsif @next_up.present? %> + <% upcoming = @next_up.first %> + <% lead = capture do %> +

<%= upcoming[:unit].code %> <%= upcoming[:task_definition].abbreviation %> <%= upcoming[:task_definition].name %>

+

is next, targeted <%= target_phrase.call(upcoming[:target_date]) %>. Nothing has slipped.

+ <% end %> + <%= render 'layouts/notification_mail/details', body: lead, category: 'due', label: 'Next up' %> +<% else %> + <%= render 'layouts/notification_mail/details', body: 'Nothing is past its target date and nothing is targeted soon in any of your units.', category: 'submission', label: 'All clear' %> +<% end %> + +<% if @needs_response_total.positive? && @show_standing %> + <% returned = capture do %> + <%= @needs_response_total %> + task<%= "s" unless @needs_response_total == 1 %> <%= @needs_response_total == 1 ? 'has' : 'have' %> come back from a tutor and <%= @needs_response_total == 1 ? 'is' : 'are' %> waiting on you to resubmit, redo or talk through. + <% end %> + <%= render 'layouts/notification_mail/details', body: returned, category: 'status:fix-and-resubmit', label: 'Feedback waiting for you' %> +<% end %> + +<% if @show_deadlines && @next_up.present? %> + <%# The one list that makes combining the units worth doing: every deadline in + date order, whichever unit it belongs to. %> +

What to do next

+ + <% @next_up.each_with_index do |entry, index| %> + <% definition = entry[:task_definition] %> + <% status_category = notification_mail_status_category(entry[:status]) %> + <% overdue_row = entry[:reason] == :overdue %> + + + + + <% end %> +
+ <%= render 'layouts/notification_mail/code_chip', code: definition.abbreviation %> + + <%= definition.name %> +
+ + <%= entry[:unit].code %> + · Target "><%= target_phrase.call(entry[:target_date]) %><% if status_category.present? %> + · <%= entry[:status].to_s.tr('_', ' ').capitalize %><% end %> + +
+<% end %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/home", label: "Open #{@doubtfire_product_name}" %> + +<% if @show_standing %> +

Across your units

+ <%= render 'layouts/notification_mail/bar_chart', scale: @grade_task_total, rows: [ + ['Complete', @tasks_complete_total, notification_mail_accent('status:complete')], + ['With a tutor to mark', @waiting_on_tutor_total, notification_mail_accent('status:ready-for-feedback')], + ['Handed back to you', @needs_response_total, notification_mail_accent('status:fix-and-resubmit')], + ['Past target date, open', @overdue_total, notification_mail_accent('status:time-exceeded')] + ] %> + + <%# One block per unit. Each names its own tutor with a mailto, because a + digest spanning several units cannot put any one of them in Reply-To. %> + <% @units.each do |summary| %> +

<%= summary[:unit].code %> <%= summary[:unit].name %>

+ + <% if @show_deadlines && (summary[:overdue].present? || summary[:soon].present?) %> +

+ <% if summary[:behind_target].positive? %> + <%= summary[:behind_target] %> past target date and open. + <% end %> + <% if summary[:soon].present? %> + <%= summary[:soon].count %> targeted in the next week. + <% end %> +

+ <% end %> + + <%= render 'layouts/notification_mail/progress', value: summary[:tasks_complete], total: summary[:grade_task_total], accent: notification_mail_accent('status:complete') %> +

+ <%= summary[:tasks_complete] %> of <%= summary[:grade_task_total] %> tasks complete for your + <%= summary[:project].target_grade_desc %> target. + <% if @show_trend %> + <%= summary[:completed_in_window] %> finished <%= @window[:since] %>. + <% end %> +

+ + <% if @show_trend && summary[:status_summary].present? %> + <%= render 'layouts/notification_mail/bar_chart', scale: summary[:grade_task_total], rows: summary[:status_summary].map { |status_key, label, count| + [label.to_s.capitalize, count, notification_mail_accent(notification_mail_status_category(status_key) || '')] + } %> + <% end %> + +

+ Open <%= summary[:unit].code %> + <% if summary[:tutor].present? && summary[:tutor].email.present? %> + · Your tutor is + <%= summary[:tutor].name %> + <% end %> +

+ <% end %> +<% end %> + +<% if @show_standing && @comments_in_window.zero? %> + <%= render 'layouts/notification_mail/note', body: "You posted no comments #{@window[:since]}. Asking a tutor a question is the fastest way to get unstuck." %> +<% end %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team +

+ +<% content_for :footer do %> +

+ You are getting this <%= @cadence %> because that is your digest setting. + You can change it or turn it off in + your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/student_digest.text.erb b/app/views/notifications_mailer/student_digest.text.erb new file mode 100644 index 0000000000..2f6fca2eb0 --- /dev/null +++ b/app/views/notifications_mailer/student_digest.text.erb @@ -0,0 +1,104 @@ +<% + target_phrase = lambda do |date| + next nil if date.blank? + + days = (date.to_date - Time.zone.today).to_i + when_text = + if days.negative? + "#{days.abs} day#{'s' unless days.abs == 1} past" + elsif days.zero? + 'today' + elsif days == 1 + 'tomorrow' + else + "in #{days} days" + end + "#{date.to_date.strftime('%a %-d %b')}, #{when_text}" + end + bar = ->(value, scale) { ('#' * [(value.to_i * 24.0 / [scale.to_i, 1].max).round, 0].max).ljust(24, '.') } +-%> +Hi <%= @student.first_name %>, + +YOUR <%= @cadence.upcase %> DIGEST +<%= @units.count %> unit<%= "s" unless @units.count == 1 %> +<% if @show_trend -%> + +YOUR MONTH +<%= @completed_in_window %> task<%= "s" unless @completed_in_window == 1 %> finished <%= @window[:since] %>, across <%= @units.count %> unit<%= "s" unless @units.count == 1 %>. +<% if @remaining_total.positive? -%> +<%= @remaining_total %> still to go.<% if @projected_shortfall.present? && @projected_shortfall.positive? %> At that rate you finish about <%= @projected_shortfall %> short before your earliest unit ends.<% elsif @projected_finish.present? %> At that rate you finish with room to spare.<% end %> +<% else -%> +Nothing is left outstanding across your units. +<% end -%> +<% elsif @overdue_total.positive? -%> + +START HERE +<%= @overdue_total %> task<%= "s" unless @overdue_total == 1 %> across your units <%= @overdue_total == 1 ? 'is' : 'are' %> past <%= @overdue_total == 1 ? 'its' : 'their' %> target date. +<% if @next_up.present? -%> +Start with <%= @next_up.first[:unit].code %> <%= @next_up.first[:task_definition].abbreviation %>, targeted <%= target_phrase.call(@next_up.first[:target_date]) %>. +<% end -%> +<% elsif @next_up.present? -%> + +NEXT UP +<%= @next_up.first[:unit].code %> <%= @next_up.first[:task_definition].abbreviation %> <%= @next_up.first[:task_definition].name %> is next, targeted <%= target_phrase.call(@next_up.first[:target_date]) %>. Nothing has slipped. +<% else -%> + +ALL CLEAR +Nothing is past its target date and nothing is targeted soon in any of your units. +<% end -%> +<% if @needs_response_total.positive? && @show_standing -%> + +FEEDBACK WAITING FOR YOU +<%= @needs_response_total %> task<%= "s" unless @needs_response_total == 1 %> <%= @needs_response_total == 1 ? 'has' : 'have' %> come back from a tutor and <%= @needs_response_total == 1 ? 'is' : 'are' %> waiting on you to resubmit, redo or talk through. +<% end -%> +<% if @show_deadlines && @next_up.present? -%> + +WHAT TO DO NEXT +<% @next_up.each do |entry| -%> +* <%= entry[:unit].code %> <%= entry[:task_definition].abbreviation %> - <%= entry[:task_definition].name %> + Target <%= target_phrase.call(entry[:target_date]) %> - <%= entry[:status].to_s.tr('_', ' ').capitalize %> + <%= @doubtfire_host %>/projects/<%= entry[:project].id %>/dashboard/<%= ERB::Util.url_encode(entry[:task_definition].abbreviation) %> +<% end -%> +<% end -%> + +Open <%= @doubtfire_product_name %>: +<%= @doubtfire_host %>/home +<% if @show_standing -%> + +ACROSS YOUR UNITS +Complete <%= bar.call(@tasks_complete_total, @grade_task_total) %> <%= @tasks_complete_total %> of <%= @grade_task_total %> +With a tutor to mark <%= bar.call(@waiting_on_tutor_total, @grade_task_total) %> <%= @waiting_on_tutor_total %> of <%= @grade_task_total %> +Handed back to you <%= bar.call(@needs_response_total, @grade_task_total) %> <%= @needs_response_total %> of <%= @grade_task_total %> +Past target date, open <%= bar.call(@overdue_total, @grade_task_total) %> <%= @overdue_total %> of <%= @grade_task_total %> +<% @units.each do |summary| -%> + +<%= summary[:unit].code %> <%= summary[:unit].name %> +<% if @show_deadlines && summary[:behind_target].positive? -%> +<%= summary[:behind_target] %> past target date and open.<% if summary[:soon].present? %> <%= summary[:soon].count %> targeted in the next week.<% end %> +<% elsif @show_deadlines && summary[:soon].present? -%> +<%= summary[:soon].count %> targeted in the next week. +<% end -%> +<%= summary[:tasks_complete] %> of <%= summary[:grade_task_total] %> tasks complete for your <%= summary[:project].target_grade_desc %> target.<% if @show_trend %> <%= summary[:completed_in_window] %> finished <%= @window[:since] %>.<% end %> +<% if @show_trend && summary[:status_summary].present? -%> +<% summary[:status_summary].each do |_key, label, count| -%> + <%= label.to_s.capitalize.ljust(32) %> <%= bar.call(count, summary[:grade_task_total]) %> <%= count %> of <%= summary[:grade_task_total] %> +<% end -%> +<% end -%> +Open <%= summary[:unit].code %>: <%= @doubtfire_host %>/projects/<%= summary[:project].id %>/dashboard +<% if summary[:tutor].present? && summary[:tutor].email.present? -%> +Your tutor is <%= summary[:tutor].name %> <<%= summary[:tutor].email %>> +<% end -%> +<% end -%> +<% end -%> +<% if @show_standing && @comments_in_window.zero? -%> + +You posted no comments <%= @window[:since] %>. Asking a tutor a question is the fastest way to get unstuck. +<% end -%> + +Cheers, +The <%= @doubtfire_product_name %> Team + +--- + +You are getting this <%= @cadence %> because that is your digest setting. +Change it or turn it off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/task_comment_created.html.erb b/app/views/notifications_mailer/task_comment_created.html.erb new file mode 100644 index 0000000000..d802b8f110 --- /dev/null +++ b/app/views/notifications_mailer/task_comment_created.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>comment<% end %> +<% content_for :preheader do %>Open the task in <%= @doubtfire_product_name %> to read it and reply.<% end %> +<% content_for :heading do %>New comment on a task<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'comment', label: 'New comment' %> +<% if @notification_url.present? %> + <%= render 'layouts/notification_mail/button', url: @notification_url, label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "The comment is not included in this email. Open the task in #{@doubtfire_product_name} to read it and reply." %> +<% content_for :footer do %> +

+ You are receiving this because your feedback notifications are turned on. + You can change that at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/task_comment_created.text.erb b/app/views/notifications_mailer/task_comment_created.text.erb new file mode 100644 index 0000000000..cc06ea3787 --- /dev/null +++ b/app/views/notifications_mailer/task_comment_created.text.erb @@ -0,0 +1,12 @@ +Hi <%= @user.name %>, + +NEW COMMENT +<%= @notification.message %> + +The comment is not included in this email. Open the task in <%= @doubtfire_product_name %> to read it and reply. +<% if @notification_url.present? -%> + +Open the task: <%= @notification_url %> +<% end -%> + +You can turn these emails off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/task_due_date_changed.html.erb b/app/views/notifications_mailer/task_due_date_changed.html.erb new file mode 100644 index 0000000000..5a4b2356f0 --- /dev/null +++ b/app/views/notifications_mailer/task_due_date_changed.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>change<% end %> +<% content_for :preheader do %>A task due date has changed.<% end %> +<% content_for :heading do %>Task due date changed<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'change', label: 'What changed' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "The new due date is not included in this email. Open the task in #{@doubtfire_product_name} to see it." %> +<% content_for :footer do %> +

+ You are receiving this because your task notifications are turned on. + You can change that at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/task_due_date_changed.text.erb b/app/views/notifications_mailer/task_due_date_changed.text.erb new file mode 100644 index 0000000000..69bc767c6d --- /dev/null +++ b/app/views/notifications_mailer/task_due_date_changed.text.erb @@ -0,0 +1,12 @@ +Hi <%= @user.name %>, + +WHAT CHANGED +<%= @notification.message %> + +The new due date is not included in this email. Open the task in <%= @doubtfire_product_name %> to see it. +<% if @notification.link.present? -%> + +Open the task: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +You can turn these emails off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/task_due_soon.html.erb b/app/views/notifications_mailer/task_due_soon.html.erb new file mode 100644 index 0000000000..481de4f0e3 --- /dev/null +++ b/app/views/notifications_mailer/task_due_soon.html.erb @@ -0,0 +1,16 @@ +<% content_for :icon do %>due<% end %> +<% content_for :preheader do %>A task is due soon.<% end %> +<% content_for :heading do %>Task due soon<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'due', label: 'Due soon' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "The deadline is not included in this email. Open the task in #{@doubtfire_product_name} to see when it is due and what is left to do." %> +<% content_for :footer do %> +

+ You are receiving this because your task notifications are turned on. + You can change that at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/task_due_soon.text.erb b/app/views/notifications_mailer/task_due_soon.text.erb new file mode 100644 index 0000000000..aa4d2c6dd8 --- /dev/null +++ b/app/views/notifications_mailer/task_due_soon.text.erb @@ -0,0 +1,13 @@ +Hi <%= @user.name %>, + +DUE SOON +<%= @notification.message %> + +The deadline is not included in this email. Open the task in <%= @doubtfire_product_name %> to see when it is due and what is left to do. +<% if @notification.link.present? -%> + +Open the task: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +You are receiving this because your task notifications are turned on. +You can turn these emails off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/task_help_requested.html.erb b/app/views/notifications_mailer/task_help_requested.html.erb new file mode 100644 index 0000000000..7606f0d7bd --- /dev/null +++ b/app/views/notifications_mailer/task_help_requested.html.erb @@ -0,0 +1,18 @@ +<% content_for :icon do %>help<% end %> +<% content_for :preheader do %><%= @notification.message.to_s.truncate(90) %><% end %> +<% content_for :heading do %>A student asked for help<% end %> + +

Hi <%= @user.first_name %>,

+

A student moved a task to Need Help and is waiting to hear from you.

+<%= render 'layouts/notification_mail/details', body: @notification.message %> + +

The student's work and comments are not included in this email.

+<% if @notification_url.present? %> + <%= render 'layouts/notification_mail/button', url: @notification_url, label: "Open the task in #{@doubtfire_product_name}" %> +<% end %> +<% content_for :footer do %> +

+ You can manage your notification preferences in + your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/task_help_requested.text.erb b/app/views/notifications_mailer/task_help_requested.text.erb new file mode 100644 index 0000000000..fb1619a6d5 --- /dev/null +++ b/app/views/notifications_mailer/task_help_requested.text.erb @@ -0,0 +1,13 @@ +Hi <%= @user.first_name %>, + +<%= @notification.message %> + +Open the task to see where they are up to: +<% if @notification_url.present? -%> +<%= @notification_url %> +<% end -%> + +The student's work and comments are not included in this email. + +You can manage your notification preferences here: +<%= @unsubscribe_url %> diff --git a/app/views/notifications_mailer/task_status_changed.html.erb b/app/views/notifications_mailer/task_status_changed.html.erb new file mode 100644 index 0000000000..684b236ffc --- /dev/null +++ b/app/views/notifications_mailer/task_status_changed.html.erb @@ -0,0 +1,23 @@ +<%# The notifiable is the task itself, so this message can carry the colour of + the status it is announcing and match what the student sees on the task in + the app. It reads the task's current status, which is the announced one in + all but the rare case where the status moves again before the mail is sent. + Anything unrecognised falls back to the neutral feedback styling. %> +<% status_category = @notification.notifiable.respond_to?(:status) ? notification_mail_status_category(@notification.notifiable.status) : nil %> +<% status_category ||= 'feedback' %> +<% content_for :icon do %><%= status_category %><% end %> +<% content_for :preheader do %>The status of one of your tasks has changed.<% end %> +<% content_for :heading do %>Task status changed<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: status_category, label: 'What changed' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: 'Open the task' %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "The new status is not included in this email. Open the task in #{@doubtfire_product_name} to see it." %> +<% content_for :footer do %> +

+ You are receiving this because your task notifications are turned on. + You can change that at your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/task_status_changed.text.erb b/app/views/notifications_mailer/task_status_changed.text.erb new file mode 100644 index 0000000000..1c9b5640c6 --- /dev/null +++ b/app/views/notifications_mailer/task_status_changed.text.erb @@ -0,0 +1,12 @@ +Hi <%= @user.name %>, + +WHAT CHANGED +<%= @notification.message %> + +The new status is not included in this email. Open the task in <%= @doubtfire_product_name %> to see it. +<% if @notification.link.present? -%> + +Open the task: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> + +You can turn these emails off at <%= @unsubscribe_url %>. diff --git a/app/views/notifications_mailer/task_submitted.html.erb b/app/views/notifications_mailer/task_submitted.html.erb new file mode 100644 index 0000000000..031d3c7ebc --- /dev/null +++ b/app/views/notifications_mailer/task_submitted.html.erb @@ -0,0 +1,17 @@ +<% content_for :icon do %>ready_for_feedback<% end %> +<% content_for :preheader do %><%= @notification.message.to_s.truncate(90) %><% end %> +<% content_for :heading do %>Task submitted<% end %> + +

Hi <%= @user.first_name %>,

+

A student has submitted work that is ready for you to mark.

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'ready_for_feedback', label: 'Submission' %> +<% if @notification_url.present? %> + <%= render 'layouts/notification_mail/button', url: @notification_url, label: "Open the task in #{@doubtfire_product_name}" %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "The submission and any assessment content are not included in this email." %> +<% content_for :footer do %> +

+ You can manage your notification preferences in + your profile. +

+<% end %> diff --git a/app/views/notifications_mailer/task_submitted.text.erb b/app/views/notifications_mailer/task_submitted.text.erb new file mode 100644 index 0000000000..ff694f0fdd --- /dev/null +++ b/app/views/notifications_mailer/task_submitted.text.erb @@ -0,0 +1,14 @@ +Hi <%= @user.first_name %>, + +SUBMISSION +<%= @notification.message %> + +Open the task to review the submission: +<% if @notification_url.present? -%> +<%= @notification_url %> +<% end -%> + +The submission and any assessment content are not included in this email. + +You can manage your notification preferences here: +<%= @unsubscribe_url %> diff --git a/app/views/notifications_mailer/tutorial_changed.html.erb b/app/views/notifications_mailer/tutorial_changed.html.erb new file mode 100644 index 0000000000..8d6a2ad6fb --- /dev/null +++ b/app/views/notifications_mailer/tutorial_changed.html.erb @@ -0,0 +1,10 @@ +<% content_for :icon do %>change<% end %> +<% content_for :preheader do %>Please use the new tutorial day and time for your next class.<% end %> +<% content_for :heading do %>Your tutorial has changed<% end %> + +

Hi <%= @user.name %>,

+<%= render 'layouts/notification_mail/details', body: @notification.message, category: 'change', label: 'Your new tutorial' %> +<% if @notification.link.present? %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}#{@notification.link}", label: "Open your unit in #{@doubtfire_product_name}" %> +<% end %> +<%= render 'layouts/notification_mail/note', body: "Please use the new tutorial day and time for your next class." %> diff --git a/app/views/notifications_mailer/tutorial_changed.text.erb b/app/views/notifications_mailer/tutorial_changed.text.erb new file mode 100644 index 0000000000..70e6f0393e --- /dev/null +++ b/app/views/notifications_mailer/tutorial_changed.text.erb @@ -0,0 +1,10 @@ +Hi <%= @user.name %>, + +YOUR NEW TUTORIAL +<%= @notification.message %> + +Please use the new tutorial day and time for your next class. +<% if @notification.link.present? -%> + +Open your unit in <%= @doubtfire_product_name %>: <%= @doubtfire_host %><%= @notification.link %> +<% end -%> diff --git a/app/views/notifications_mailer/weekly_staff_summary.html.erb b/app/views/notifications_mailer/weekly_staff_summary.html.erb index c8360dfde8..7bfb0acaed 100644 --- a/app/views/notifications_mailer/weekly_staff_summary.html.erb +++ b/app/views/notifications_mailer/weekly_staff_summary.html.erb @@ -1,122 +1,102 @@ - - - - - - -
-

<%= @summary_stats[:unit].name %> - Weekly Summary

-

<%= "#{@summary_stats[:week_start].day.ordinalize} #{@summary_stats[:week_start].strftime("%B %Y")}" %> to <%= "#{@summary_stats[:week_end].day.ordinalize} #{@summary_stats[:week_end].strftime("%B %Y")}" %>

+<% section_heading = 'margin:0 0 12px 0; font-size:14px; line-height:1.4; font-weight:600; letter-spacing:0.4px; text-transform:uppercase;' %> +<% content_for :icon do %>summary<% end %> +<% week_range = "#{@summary_stats[:week_start].day.ordinalize} #{@summary_stats[:week_start].strftime("%B %Y")} to #{@summary_stats[:week_end].day.ordinalize} #{@summary_stats[:week_end].strftime("%B %Y")}" %> +<% content_for :preheader do %><%= week_range %><% end %> +<% content_for :heading do %><%= @summary_stats[:unit].name %> - Weekly Summary<% end %> +<% content_for :subheading do %><%= week_range %><% end %> + +

Hi <%= @staff.first_name %>,

-

Hi <%= @staff.first_name %>,

-

- Hope you had a good week! Here's a summary of what has happened in this unit over the last week. -

<% if @unit_role.has_students? %> -

-

-

-<% if @summary_stats[:revert][@staff].count > 0 %> -

- The following students in your tutorials have been advised to revert their grade to Pass as they are falling behind with their pass tasks: -

-

-<% end %> -<%# Else if unit role has no students %> + <%# Lead with the number a tutor has to act on. Everything else in this email + is a record of the week, this is the only part that is still outstanding. %> + <% waiting = @data[:tasks_awaiting_feedback_count].to_i %> + <% oldest = @data[:oldest_task_days].to_i %> + <% waiting_category = if waiting.zero? + 'submission' + elsif oldest >= 7 + 'due' + else + 'summary' + end %> + <% waiting_summary = capture do %> + <% if waiting.zero? %> + Nothing is waiting for your feedback. + <% else %> + <%= waiting %> + task<%= "s" unless waiting == 1 %> <%= are_is(waiting) %> waiting for your feedback. + The oldest was submitted <%= oldest %> day<%= "s" unless oldest == 1 %> ago. + <% end %> + <% end %> + <%= render 'layouts/notification_mail/details', body: waiting_summary, category: waiting_category, label: 'Waiting on you' %> + + <%# The only link in this email used to be Unsubscribe. It told a tutor how + many tasks were waiting and gave them no way to reach any of them. %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/units/#{@unit.id}/tasks/inbox", label: 'Open your marking queue' %> + +

Your week

+ <%= render 'layouts/notification_mail/stats', rows: [ + [@data[:received_comments], "comment#{'s' if @data[:received_comments] != 1} and pieces of feedback received"], + [@data[:sent_comments], "comment#{'s' if @data[:sent_comments] != 1} and pieces of feedback posted"], + [@data[:staff_engagements], "task#{'s' unless @data[:staff_engagements] == 1} you gave a new status"], + [@data[:weekly_total_tasks_discussed], "task#{'s' unless @data[:weekly_total_tasks_discussed] == 1} you discussed"], + [@data[:weekly_engagements_count], "state change#{'s' unless @data[:weekly_engagements_count] == 1} on your students' tasks"] + ] %> + +

Across the unit

+ <%= render 'layouts/notification_mail/stats', rows: [ + [@summary_stats[:unit_week_comments], "text comment#{'s' if @summary_stats[:unit_week_comments] != 1} made"], + [@summary_stats[:unit_week_engagements], "task state change#{'s' unless @summary_stats[:unit_week_engagements] == 1}"] + ] %> + + <% if @summary_stats[:revert][@staff].count > 0 %> + <% revert_list = capture do %> +

+ These students have been advised to revert their grade to Pass, because they are falling behind on their pass tasks. +

+ + <% end %> + <%= render 'layouts/notification_mail/details', body: revert_list, category: 'due', label: 'Falling behind' %> + <% end %> <% else %> -

-

-

+

Hope you had a good week. Here is what happened in this unit.

+ <%= render 'layouts/notification_mail/stats', rows: [ + [@summary_stats[:unit_week_comments], "text comment#{'s' if @summary_stats[:unit_week_comments] != 1} made in this unit"], + [@summary_stats[:unit_week_engagements], "task state change#{'s' unless @summary_stats[:unit_week_engagements] == 1} in this unit"] + ] %> <% end %> <% if @unit_role.is_convenor? && !@summary_stats[:tutorial_streams].nil? && !@summary_stats[:tutorial_streams].empty?%> <%# Create a table for each tutorial stream %> <% @summary_stats[:tutorial_streams].each do |tutorial_stream, tutorial_stream_data | %> <% next unless tutorial_stream_data[:stream_linked_to_task_definition] %> -

- Tutorial Stream: <%= tutorial_stream.name %> -

+

+ Tutorial stream: <%= tutorial_stream.name %> +

<%# Skip table if no tutorials%> <% unless tutorial_stream_data.nil? || tutorial_stream_data[:tutorials].nil?%> - +
- - - - - - - + + + + + + + - - - - - - + + + + + + @@ -126,35 +106,33 @@ <% sorted_data.each do | unit_role, data | %> <% next unless data[:number_of_students] > 0%> - - - - - - - - - - + + + + + + + + + + <% end %> -
NameStudentsAssessmentsCommentsAwaiting FeedbackOldest TaskDiscussionsNameStudentsAssessmentsCommentsAwaiting FeedbackOldest TaskDiscussions
TotalWeekTotalWeekTotalWeekTotalWeekTotalWeekTotalWeek
<%= '%-12s' % unit_role.user.name.truncate(14) %><%= '%8i' % data[:number_of_students] %><%= '%7i' % data[:total_staff_engagements] %><%= '%6i' % data[:staff_engagements] %><%= '%5i' % data[:total_comments].uniq.count %><%= '%5i' % data[:sent_comments].uniq.count %><%= '%8i' % data[:tasks_awaiting_feedback_count] %>><%= '%6i' % data[:oldest_task_days] %> days<%= '%5i' % data[:total_tasks_discussed] %><%= '%5i' % data[:weekly_tasks_discussed] %><%= '%-12s' % unit_role.user.name.truncate(14) %><%= '%8i' % data[:number_of_students] %><%= '%7i' % data[:total_staff_engagements] %><%= '%6i' % data[:staff_engagements] %><%= '%5i' % data[:total_comments].uniq.count %><%= '%5i' % data[:sent_comments].uniq.count %><%= '%8i' % data[:tasks_awaiting_feedback_count] %><%= '%6i' % data[:oldest_task_days] %> days<%= '%5i' % data[:total_tasks_discussed] %><%= '%5i' % data[:weekly_tasks_discussed] %>
+
<% end %> <% if tutorial_stream_data[:num_students_without_tutors] > 0 %> -

- Please note that <%=tutorial_stream_data[:num_students_without_tutors]%> student<%= "s" unless tutorial_stream_data[:num_students_without_tutors] == 1 %> <%= are_is(tutorial_stream_data[:num_students_without_tutors]) %> not allocated to a tutorial in this stream. Work submitted by <%= this_these(tutorial_stream_data[:num_students_without_tutors]) %> student<%= "s" unless tutorial_stream_data[:num_students_without_tutors] == 1 %> will not appear in any of the tutor inboxes! -

+ <%= render 'layouts/notification_mail/note', body: "#{tutorial_stream_data[:num_students_without_tutors]} student#{"s" unless tutorial_stream_data[:num_students_without_tutors] == 1} #{are_is(tutorial_stream_data[:num_students_without_tutors])} not allocated to a tutorial in this stream. Work submitted by #{this_these(tutorial_stream_data[:num_students_without_tutors])} student#{"s" unless tutorial_stream_data[:num_students_without_tutors] == 1} will not appear in any of the tutor inboxes." %> <% end %> <% end %> <% end %> -

+

Cheers,
The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %>

- - - - + +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/notifications_mailer/weekly_staff_summary.text.erb b/app/views/notifications_mailer/weekly_staff_summary.text.erb index 1bdde187a8..917b34ed80 100644 --- a/app/views/notifications_mailer/weekly_staff_summary.text.erb +++ b/app/views/notifications_mailer/weekly_staff_summary.text.erb @@ -3,29 +3,38 @@ Hi <%= @staff.first_name %>, <%= @summary_stats[:unit].name %> - Weekly Summary <%= "#{@summary_stats[:week_start].day.ordinalize} #{@summary_stats[:week_start].strftime("%B %Y")}" %> to <%= "#{@summary_stats[:week_end].day.ordinalize} #{@summary_stats[:week_end].strftime("%B %Y")}" %> -Hope you had a good week! Here's a summary of what has happened in this unit over the last week. - <% if @unit_role.has_students? %> -* In total, there <%= were_was(@summary_stats[:unit_week_comments]) %> <%= @summary_stats[:unit_week_comments] %> comment<%= "s" if @summary_stats[:unit_week_comments] != 1 %> made this unit. -* You received <%= @data[:received_comments] %> comment<%= "s" if @data[:received_comments] != 1 %> and posted a total of <%= @data[:sent_comments] %> comment<%= "s" if @data[:sent_comments] != 1 %>. -* Tasks changed state <%= @summary_stats[:unit_week_engagements] %> time<%= "s" unless @summary_stats[:unit_week_engagements] == 1 %> in this unit. -* Your student's tasks changed state <%= @data[:weekly_engagements_count] %> time<%= "s" unless @data[:weekly_engagements_count] == 1 %>. -* You assigned a new status to <%= @data[:staff_engagements] %> task<%= "s" unless @data[:staff_engagements] == 1 %>. -* You have discussed <%= @data[:weekly_total_tasks_discussed] %> task<%= "s" unless @data[:weekly_total_tasks_discussed] == 1 %>. -* You have <%= @data[:tasks_awaiting_feedback_count] %> task<%= "s" unless @data[:tasks_awaiting_feedback_count] == 1 %> waiting for feedback. -<% if @data[:tasks_awaiting_feedback_count] > 0 %> -* The oldest of these tasks was submitted <%= @data[:oldest_task_days] %> day<%= "s" unless @data[:oldest_task_days] == 1 %> ago. -<% end %> +WAITING ON YOU +<% if @data[:tasks_awaiting_feedback_count].to_i.zero? %> +Nothing is waiting for your feedback. +<% else %> +<%= @data[:tasks_awaiting_feedback_count] %> task<%= "s" unless @data[:tasks_awaiting_feedback_count] == 1 %> <%= are_is(@data[:tasks_awaiting_feedback_count]) %> waiting for your feedback. The oldest was submitted <%= @data[:oldest_task_days] %> day<%= "s" unless @data[:oldest_task_days] == 1 %> ago. +<% end %> +Open your marking queue: +<%= @doubtfire_host %>/units/<%= @unit.id %>/tasks/inbox + +YOUR WEEK +* <%= @data[:received_comments] %> comment<%= "s" if @data[:received_comments] != 1 %> and pieces of feedback received +* <%= @data[:sent_comments] %> comment<%= "s" if @data[:sent_comments] != 1 %> and pieces of feedback posted +* <%= @data[:staff_engagements] %> task<%= "s" unless @data[:staff_engagements] == 1 %> you gave a new status +* <%= @data[:weekly_total_tasks_discussed] %> task<%= "s" unless @data[:weekly_total_tasks_discussed] == 1 %> you discussed +* <%= @data[:weekly_engagements_count] %> state change<%= "s" unless @data[:weekly_engagements_count] == 1 %> on your students' tasks + +ACROSS THE UNIT +* <%= @summary_stats[:unit_week_comments] %> text comment<%= "s" if @summary_stats[:unit_week_comments] != 1 %> made +* <%= @summary_stats[:unit_week_engagements] %> task state change<%= "s" unless @summary_stats[:unit_week_engagements] == 1 %> <% if @summary_stats[:revert][@staff].count > 0 %> -The following students in your tutorials have been advised to revert their grade to Pass as they are falling behind with their pass tasks: +FALLING BEHIND +These students have been advised to revert their grade to Pass, because they are falling behind on their pass tasks: <% @summary_stats[:revert][@staff].each do |p|%> * <%= p.student.name %> <% end %> <% end %> <% else %> -* In total, there <%= were_was(@summary_stats[:unit_week_comments]) %> <%= @summary_stats[:unit_week_comments] %> comment<%= "s" if @summary_stats[:unit_week_comments] != 1 %> made in this unit. -* Tasks changed state <%= @summary_stats[:unit_week_engagements] %> time<%= "s" unless @summary_stats[:unit_week_engagements] == 1 %> in this unit. +ACROSS THE UNIT +* <%= @summary_stats[:unit_week_comments] %> text comment<%= "s" if @summary_stats[:unit_week_comments] != 1 %> made in this unit +* <%= @summary_stats[:unit_week_engagements] %> task state change<%= "s" unless @summary_stats[:unit_week_engagements] == 1 %> in this unit <% end %> <% if @unit_role.is_convenor? && !@summary_stats[:tutorial_streams].nil? && !@summary_stats[:tutorial_streams].empty?%> diff --git a/app/views/notifications_mailer/weekly_student_summary.html.erb b/app/views/notifications_mailer/weekly_student_summary.html.erb index 3a0593d36e..436717d0c8 100644 --- a/app/views/notifications_mailer/weekly_student_summary.html.erb +++ b/app/views/notifications_mailer/weekly_student_summary.html.erb @@ -1,118 +1,210 @@ - - - - - - -
-

<%= @summary_stats[:unit].name %> - Weekly Summary

-

<%= "#{@summary_stats[:week_start].day.ordinalize} #{@summary_stats[:week_start].strftime("%B %Y")}" %> to <%= "#{@summary_stats[:week_end].day.ordinalize} #{@summary_stats[:week_end].strftime("%B %Y")}" %>

- -

Hi <%= @student.first_name %>,

+<% + week_range = "#{@summary_stats[:week_start].day.ordinalize} #{@summary_stats[:week_start].strftime("%B %Y")} to #{@summary_stats[:week_end].day.ordinalize} #{@summary_stats[:week_end].strftime("%B %Y")}" + overdue_count = @overdue_top.to_a.count + soon_count = @soon_top.to_a.count + + # The due date top_tasks sorted on, matched back by task definition. A task the + # student has not opened yet has no row of its own, so it falls back to the + # date the definition targets. + due_date_for = lambda do |definition| + @task_due_dates[definition.id] || definition.target_date + end + + # "Fri 3 Oct, 4 days late" rather than a bare date, because the number of days + # is the part that decides what a student does next. + due_phrase = lambda do |date| + return nil if date.blank? + + days = (date.to_date - Time.zone.today).to_i + when_text = + if days.negative? + "#{days.abs} day#{'s' unless days.abs == 1} past" + elsif days.zero? + 'today' + elsif days == 1 + 'tomorrow' + else + "in #{days} days" + end + "#{date.to_date.strftime('%a %-d %b')}, #{when_text}" + end + + next_due = (@overdue_top.to_a + @soon_top.to_a + @ahead_top.to_a).first + section_heading = 'margin:0 0 12px 0; font-size:14px; font-weight:600; letter-spacing:0.4px; text-transform:uppercase;' +%> +<% content_for :icon do %><%= overdue_count.positive? ? 'overdue' : 'summary' %><% end %> +<% content_for :preheader do %><%= overdue_count.positive? ? "#{@overdue_top.first[:task_definition].abbreviation} is overdue." : "#{@tasks_complete} of #{@grade_task_total} tasks complete." %> <%= week_range %><% end %> +<% content_for :heading do %><%= @summary_stats[:unit].name %> - Weekly Summary<% end %> +<% content_for :subheading do %><%= week_range %><% end %> + +

Hi <%= @student.first_name %>,

+ <% if @did_revert_to_pass %> -

Hope you had a good week!

-

Before we get into the summary, it seems that you are falling behind in your Pass Tasks. It is really important that you catch up with these tasks, as you must have all Pass tasks marked as Complete to Pass the unit. I've reset your Target Grade to Pass for the moment, to help you focus on these tasks. I would like to encourage you to work through the Pass tasks in order to catch up as quickly as you can. Once you have caught up, you can upgrade your Target Grade again, and go back and complete any higher grade tasks you skipped.

-

With that out of the way, here's a summary of what has happened in this unit over the last week and some notes on what you should do next.

-<% else %> -

Hope you had a good week! Here's a summary of what has happened in this unit over the last week and some notes on what you should do next.

+ <% revert_note = capture do %> +

Your target grade has been reset to Pass while you catch up.

+

Every Pass task has to be marked Complete to pass this unit, so work through them in order. Once you have caught up you can set your target grade back and return to the higher grade tasks you skipped.

+ <% end %> + <%= render 'layouts/notification_mail/details', body: revert_note, category: 'overdue', label: 'Falling behind on pass tasks' %> <% end %> <% if @project.tutorial_enrolments.blank? %> -

Firstly... it looks like you are not assigned a tutor! Please login and make sure your tutorial is correctly set. You should be able to do that on the tutorials page.

+ <% no_tutor = capture do %> + You are not assigned a tutor. Set your tutorial on your + tutorials page + so your work reaches someone. + <% end %> + <%= render 'layouts/notification_mail/details', body: no_tutor, category: 'alert', label: 'No tutor set' %> +<% end %> + +<%# top_tasks hands back at most five rows in total, so the number of overdue + ones it carries is a floor and not a count. The lead names the single oldest + instead, which is both the honest claim and the one a student acts on. %> +<% if overdue_count.positive? %> + <% oldest = @overdue_top.first[:task_definition] %> + <% focus = capture do %> +

<%= oldest.abbreviation %> <%= oldest.name %>

+

is the one furthest past its target date, targeted <%= due_phrase.call(due_date_for.call(oldest)) %>. Start there.

+ <% end %> + <%= render 'layouts/notification_mail/details', body: focus, category: 'overdue', label: 'Start here' %> +<% elsif @behind_target.to_i.positive? %> + <%# Nothing reached the list above, but tasks have still slipped past their + target date. They are in a state the deadline list does not select for, so + saying "nothing is overdue" here would contradict the chart below. %> + <% slipped = capture do %> +

<%= @behind_target %> + task<%= "s" unless @behind_target == 1 %> <%= @behind_target == 1 ? 'is' : 'are' %> past + <%= @behind_target == 1 ? 'its' : 'their' %> target date and not finished. None of them is + on the list below, so open your dashboard to find them.

+ <% end %> + <%= render 'layouts/notification_mail/details', body: slipped, category: 'overdue', label: 'Behind the target schedule' %> +<% elsif next_due.present? %> + <% focus = capture do %> +

<%= next_due[:task_definition].abbreviation %> <%= next_due[:task_definition].name %>

+

is next, targeted <%= due_phrase.call(due_date_for.call(next_due[:task_definition])) %>. Nothing has slipped.

+ <% end %> + <%= render 'layouts/notification_mail/details', body: focus, category: soon_count.positive? ? 'due' : 'summary', label: 'Next target date' %> +<%# Nothing claims the student is done unless the counts below agree. top_tasks + covers only the statuses it selects for, so it going empty is not on its own + proof that there is no work left, and an email must not contradict its own + charts a screen later. %> +<% elsif @waiting_on_student.to_i.zero? && @project.portfolio_exists? %> + <%= render 'layouts/notification_mail/details', body: 'Every task is done and your portfolio is prepared. Nothing is outstanding.', category: 'submission', label: 'All caught up' %> +<% elsif @waiting_on_student.to_i.zero? %> + <%= render 'layouts/notification_mail/details', body: 'Every task is done. Prepare your portfolio to finish up.', category: 'portfolio', label: 'Almost there' %> +<% else %> + <% steady = capture do %> +

Nothing has slipped past its target date and nothing is targeted this week. + <%= @waiting_on_student %> task<%= "s" unless @waiting_on_student == 1 %> <%= @waiting_on_student == 1 ? 'is' : 'are' %> still with you, so this is a good week to get ahead.

+ <% end %> + <%= render 'layouts/notification_mail/details', body: steady, category: 'summary', label: 'Nothing targeted this week' %> +<% end %> + +<%# Each task with its own date and its own status colour, so the list answers + what is late, what is next and whose move it is without a legend. %> +<% task_groups = [ + ['Past target date', @overdue_top, 'time-exceeded'], + ['Target date this week', @soon_top, 'working-on-it'], + ['Get ahead', @ahead_top, nil] + ] %> +<% task_groups.each do |group_label, group_tasks, group_status| %> + <% next if group_tasks.blank? %> +

;"><%= group_label %>

+ + <% group_tasks.each_with_index do |tt, index| %> + <% definition = tt[:task_definition] %> + <% status_category = notification_mail_status_category(tt[:status]) %> + + + + + <% end %> +
+ <%= render 'layouts/notification_mail/code_chip', code: definition.abbreviation %> + + <%= definition.name %> +
+ + Target <%= due_phrase.call(due_date_for.call(definition)) %><% if status_category.present? %> · <%= tt[:status].to_s.tr('_', ' ').capitalize %><% end %> + +
+<% end %> + +<% if @top_tasks_truncated %> + <%= render 'layouts/notification_mail/note', body: "These are the #{NotificationsMailer::TOP_TASK_LIMIT} most urgent. Your dashboard has the rest." %> +<% end %> + +<% if @needs_your_response.to_i.positive? %> + <% returned = capture do %> + <%= @needs_your_response %> + task<%= "s" unless @needs_your_response == 1 %> <%= @needs_your_response == 1 ? 'has' : 'have' %> come back from your tutor and <%= @needs_your_response == 1 ? 'is' : 'are' %> waiting on you to resubmit, redo or talk it through. + <% end %> + <%= render 'layouts/notification_mail/details', body: returned, category: 'status:fix-and-resubmit', label: 'Feedback waiting for you' %> +<% end %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/dashboard", label: "Open your unit in #{@doubtfire_product_name}" %> + +<%# Where the student stands against the grade they are aiming at, drawn rather + than listed. The comment counts this block used to carry were a scoreboard, + not something anyone could act on. %> +<% if @grade_task_total.to_i.positive? %> +

Your <%= @project.target_grade_desc %> target

+ <%= render 'layouts/notification_mail/progress', value: @tasks_complete, total: @grade_task_total, accent: notification_mail_accent('status:complete') %> +

+ <%= @tasks_complete %> of <%= @grade_task_total %> tasks complete. + <% if @behind_target.to_i.positive? %> + <%= @behind_target %> of the <%= @targets_passed %> whose target date has passed <%= @behind_target == 1 ? 'is' : 'are' %> still open. + <% elsif @targets_passed.to_i.positive? %> + All <%= @targets_passed %> whose target date has passed are done. + <% end %> +

+ +

Pace

+ <%= render 'layouts/notification_mail/bar_chart', scale: @grade_task_total, rows: [ + ['Target date passed', @targets_passed, notification_mail_accent('status:not-started')], + ['Of those, finished', @on_target, notification_mail_accent('status:complete')] + ] %> +<% end %> + +<% if @status_summary.present? %> +

Every task, by status

+ <%= render 'layouts/notification_mail/bar_chart', scale: @grade_task_total, rows: @status_summary.map { |status_key, label, count| + [label.to_s.capitalize, count, notification_mail_accent(notification_mail_status_category(status_key) || '')] + } %> +<% end %> + +

Whose move

+<% standing_rows = [ + [@waiting_on_tutor, 'with your tutor, waiting to be marked', notification_mail_accent('status:ready-for-feedback')], + [@needs_your_response, 'handed back for you to act on', notification_mail_accent('status:fix-and-resubmit')], + [@waiting_on_student, 'still with you in total', notification_mail_accent('status:working-on-it')] + ] %> +<%# A count of comments posted is not something a student can act on, so it only + appears when it is nought, where it is a prompt rather than a scoreboard. %> +<% standing_rows << [0, 'comments posted this week - asking your tutor a question is the fastest way to get unstuck'] if @sent_comments.zero? %> +<%= render 'layouts/notification_mail/stats', rows: standing_rows %> + +<% if @weeks_left.present? && @weeks_left.positive? %> + <%= render 'layouts/notification_mail/note', body: "#{@weeks_left} week#{'s' unless @weeks_left == 1} left in this unit." %> +<% end %> + +<% if @project.portfolio_exists? %> + <%= render 'layouts/notification_mail/note', body: "Your portfolio is prepared.#{@portfolio_days_left.present? && @portfolio_days_left >= 0 ? " You can keep changing it for another #{@portfolio_days_left} day#{'s' unless @portfolio_days_left == 1}." : ''}" %> +<% elsif @portfolio_days_left.present? && @portfolio_days_left.between?(0, 42) %> + <% portfolio_note = capture do %> + Your portfolio is not prepared yet and the deadline is + <%= @portfolio_due.to_date.strftime('%a %-d %b') %>, + <%= @portfolio_days_left.zero? ? 'today' : "in #{@portfolio_days_left} day#{'s' unless @portfolio_days_left == 1}" %>. + <% end %> + <%= render 'layouts/notification_mail/details', body: portfolio_note, category: @portfolio_days_left <= 14 ? 'due' : 'portfolio', label: 'Portfolio' %> +<% end %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

<% end %> -

Here is what you should focus on right now.

- -<% if @top_tasks && @top_tasks.count > 0 %> -<% if @overdue_top && @overdue_top.count > 0 %> -

Catch up by completing the following overdue task<%="s" if @overdue_top.count > 1%>!

- -<% end %> -<% if @soon_top && @soon_top.count > 0 %> -

Work to get the following task<%="s" if @soon_top.count > 1%> done, as these are due soon!

- -<% end %> -<% if @ahead_top && @ahead_top.count > 0 %> -

Get ahead by working on the following task<%="s" if @ahead_top.count > 1%> next!

- -<% end %> -<% elsif @project.portfolio_exists? %> -

Its time to Party! You have completed all of the tasks and prepared your portfolio.

-<% else %> -

Its almost party time... You have completed all of the tasks, now make sure you login and prepare your portfolio.

-<% end %> -

- What has happened in <%= @doubtfire_product_name %> this week: -

- We hope your studies are going well, and look forward to your submissions over the next week! -

- -

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> -

-
- - - diff --git a/app/views/notifications_mailer/weekly_student_summary.text.erb b/app/views/notifications_mailer/weekly_student_summary.text.erb index 808fc7c3bd..8379a55b6c 100644 --- a/app/views/notifications_mailer/weekly_student_summary.text.erb +++ b/app/views/notifications_mailer/weekly_student_summary.text.erb @@ -1,61 +1,128 @@ +<% + overdue_count = @overdue_top.to_a.count + due_date_for = ->(definition) { @task_due_dates[definition.id] || definition.target_date } + due_phrase = lambda do |date| + next nil if date.blank? + + days = (date.to_date - Time.zone.today).to_i + when_text = + if days.negative? + "#{days.abs} day#{'s' unless days.abs == 1} past" + elsif days.zero? + 'today' + elsif days == 1 + 'tomorrow' + else + "in #{days} days" + end + "#{date.to_date.strftime('%a %-d %b')}, #{when_text}" + end + next_due = (@overdue_top.to_a + @soon_top.to_a + @ahead_top.to_a).first + bar = ->(value) { ('#' * [(value.to_i * 24.0 / [@grade_task_total.to_i, 1].max).round, 0].max).ljust(24, '.') } +-%> Hi <%= @student.first_name %>, <%= @summary_stats[:unit].name %> - Weekly Summary <%= "#{@summary_stats[:week_start].day.ordinalize} #{@summary_stats[:week_start].strftime("%B %Y")}" %> to <%= "#{@summary_stats[:week_end].day.ordinalize} #{@summary_stats[:week_end].strftime("%B %Y")}" %> - -<% if @did_revert_to_pass %> -Hope you had a good week! - -Before we get into the summary, it seems that you are falling behind in your Pass Tasks. It is really important that you catch up with these tasks, as you must have all Pass tasks marked as Complete to Pass the unit. I've reset your Target Grade to Pass for the moment, to help you focus on these tasks. I would like to encourage you to work through the Pass tasks in order to catch up as quickly as you can. Once you have caught up, you can upgrade your Target Grade again, and go back and complete any higher grade tasks you skipped. - -With that out of the way, here's a summary of what has happened in this unit over the last week and some notes on what you should do next. -<% else %> -Hope you had a good week! Here's a summary of what has happened in this unit over the last week and some notes on what you should do next. -<% end %> - -<% if @project.tutorial_enrolments.blank? %> - -Firstly... it looks like you are not assigned a tutor! Please login and make sure your tutorial is correctly set. You should be able to do that here: <%= @doubtfire_host %>/#/projects/<%= @project.id %>/tutorials - -Here is what you should focus on right now. -<% else %> - -Firstly, here is what you should focus on right now. -<% end %> - -<% if @top_tasks && @top_tasks.count > 0 %> -<% if @overdue_top && @overdue_top.count > 0 %> -Catch up by completing the following *overdue* task<%="s" if @overdue_top.count > 1%>! -<% @overdue_top.each do |ot| %> -* <%= top_task_desc(ot) %> -<% end %> -<% end %> -<% if @soon_top && @soon_top.count > 0 %> -Work to get the following task<%="s" if @soon_top.count > 1%> done, as these are *due* soon! -<% @soon_top.each do |st| %> -* <%= top_task_desc(st) %> -<% end %> -<% end %> -<% if @ahead_top && @ahead_top.count > 0 %> -Get ahead by working on the following task<%="s" if @ahead_top.count > 1%> next! -<% @ahead_top.each do |at| %> -* <%= top_task_desc(at) %> -<% end %> -<% end %> -<% elsif @project.portfolio_exists? %> -Its time to Party! You have completed all of the tasks and prepared your portfolio. -<% else %> -Its almost party time... You have completed all of the tasks, now make sure you login and prepare your portfolio. -<% end %> - -What has happened in <%= @doubtfire_product_name %> this week: - -* In total, there <%= were_was(@summary_stats[:unit_week_comments]) %> <%= @summary_stats[:unit_week_comments] %> comment<%= "s" if @summary_stats[:unit_week_comments] != 1 %> made in this unit. -* You posted a total of <%= @sent_comments %> comment<%= "s" if @sent_comments != 1 %>, and received back <%= @received_comments %> comment<%= "s" if @received_comments != 1 %> <%= "- try posting some comments this week" if @sent_comments == 0 %> -* Tasks changed state <%= @summary_stats[:unit_week_engagements] %> time<%= "s" unless @summary_stats[:unit_week_engagements] == 1 %> in this unit. -* Your tasks changed state <%= @engagements_count %> time<%= "s" unless @engagements_count == 1 %> <%= "- looks like you need to be more active" if @student_engagements == 0%> - -We hope your studies are going well, and look forward to your submissions over the next week! +<% if @did_revert_to_pass -%> + +FALLING BEHIND ON PASS TASKS +Your target grade has been reset to Pass while you catch up. Every Pass task has to be marked Complete to pass this unit, so work through them in order. Once you have caught up you can set your target grade back and return to the higher grade tasks you skipped. +<% end -%> +<% if @project.tutorial_enrolments.blank? -%> + +NO TUTOR SET +You are not assigned a tutor. Set your tutorial so your work reaches someone: +<%= @doubtfire_host %>/projects/<%= @project.id %>/tutorials +<% end -%> +<% if overdue_count.positive? -%> + +START HERE +<%= @overdue_top.first[:task_definition].abbreviation %> <%= @overdue_top.first[:task_definition].name %> is the one furthest past its target date, targeted <%= due_phrase.call(due_date_for.call(@overdue_top.first[:task_definition])) %>. Start there. +<% elsif @behind_target.to_i.positive? -%> + +BEHIND THE TARGET SCHEDULE +<%= @behind_target %> task<%= "s" unless @behind_target == 1 %> <%= @behind_target == 1 ? 'is' : 'are' %> past <%= @behind_target == 1 ? 'its' : 'their' %> target date and not finished. None of them is on the list below, so open your dashboard to find them. +<% elsif next_due.present? -%> + +NEXT TARGET DATE +<%= next_due[:task_definition].abbreviation %> <%= next_due[:task_definition].name %> is next, targeted <%= due_phrase.call(due_date_for.call(next_due[:task_definition])) %>. Nothing has slipped. +<% elsif @waiting_on_student.to_i.zero? && @project.portfolio_exists? -%> + +ALL CAUGHT UP +Every task is done and your portfolio is prepared. Nothing is outstanding. +<% elsif @waiting_on_student.to_i.zero? -%> + +ALMOST THERE +Every task is done. Prepare your portfolio to finish up. +<% else -%> + +NOTHING TARGETED THIS WEEK +Nothing has slipped past its target date and nothing is targeted this week. <%= @waiting_on_student %> task<%= "s" unless @waiting_on_student == 1 %> <%= @waiting_on_student == 1 ? 'is' : 'are' %> still with you, so this is a good week to get ahead. +<% end -%> +<% [['PAST TARGET DATE', @overdue_top], ['TARGET DATE THIS WEEK', @soon_top], ['GET AHEAD', @ahead_top]].each do |group_label, group_tasks| -%> +<% next if group_tasks.blank? -%> + +<%= group_label %> +<% group_tasks.each do |tt| -%> +* <%= tt[:task_definition].abbreviation %> - <%= tt[:task_definition].name %> + Target <%= due_phrase.call(due_date_for.call(tt[:task_definition])) %> - <%= tt[:status].to_s.tr('_', ' ').capitalize %> + <%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard/<%= ERB::Util.url_encode(tt[:task_definition].abbreviation) %> +<% end -%> +<% end -%> +<% if @top_tasks_truncated -%> + +These are the <%= NotificationsMailer::TOP_TASK_LIMIT %> most urgent. Your dashboard has the rest. +<% end -%> +<% if @needs_your_response.to_i.positive? -%> + +FEEDBACK WAITING FOR YOU +<%= @needs_your_response %> task<%= "s" unless @needs_your_response == 1 %> <%= @needs_your_response == 1 ? 'has' : 'have' %> come back from your tutor and <%= @needs_your_response == 1 ? 'is' : 'are' %> waiting on you to resubmit, redo or talk it through. +<% end -%> + +Open your unit in <%= @doubtfire_product_name %>: +<%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard +<% if @grade_task_total.to_i.positive? -%> + +YOUR <%= @project.target_grade_desc.to_s.upcase %> TARGET +<%= @tasks_complete %> of <%= @grade_task_total %> tasks complete. +<% if @behind_target.to_i.positive? -%> +<%= @behind_target %> of the <%= @targets_passed %> whose target date has passed <%= @behind_target == 1 ? 'is' : 'are' %> still open. +<% elsif @targets_passed.to_i.positive? -%> +All <%= @targets_passed %> whose target date has passed are done. +<% end -%> + +PACE +Target date passed <%= bar.call(@targets_passed) %> <%= @targets_passed %> of <%= @grade_task_total %> +Of those, finished <%= bar.call(@on_target) %> <%= @on_target %> of <%= @grade_task_total %> +<% end -%> +<% if @status_summary.present? -%> + +EVERY TASK, BY STATUS +<% @status_summary.each do |_status_key, label, count| -%> +<%= label.to_s.capitalize.ljust(32) %> <%= bar.call(count) %> <%= count %> of <%= @grade_task_total %> +<% end -%> +<% end -%> + +WHOSE MOVE +* <%= @waiting_on_tutor %> with your tutor, waiting to be marked +* <%= @needs_your_response %> handed back for you to act on +* <%= @waiting_on_student %> still with you in total +<% if @sent_comments.zero? -%> +* 0 comments posted this week - asking your tutor a question is the fastest way to get unstuck +<% end -%> +<% if @weeks_left.present? && @weeks_left.positive? -%> + +<%= @weeks_left %> week<%= "s" unless @weeks_left == 1 %> left in this unit. +<% end -%> +<% if @project.portfolio_exists? -%> + +Your portfolio is prepared.<%= " You can keep changing it for another #{@portfolio_days_left} day#{'s' unless @portfolio_days_left == 1}." if @portfolio_days_left.present? && @portfolio_days_left >= 0 %> +<% elsif @portfolio_days_left.present? && @portfolio_days_left.between?(0, 42) -%> + +PORTFOLIO +Your portfolio is not prepared yet and the deadline is <%= @portfolio_due.to_date.strftime('%a %-d %b') %>, <%= @portfolio_days_left.zero? ? 'today' : "in #{@portfolio_days_left} day#{'s' unless @portfolio_days_left == 1}" %>. +<% end -%> Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> diff --git a/app/views/portfolio_evidence_mailer/overseer_assessment_failed.html.erb b/app/views/portfolio_evidence_mailer/overseer_assessment_failed.html.erb index 95fcda7a4e..333b0073b2 100644 --- a/app/views/portfolio_evidence_mailer/overseer_assessment_failed.html.erb +++ b/app/views/portfolio_evidence_mailer/overseer_assessment_failed.html.erb @@ -1,72 +1,23 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

-

Hi <%= @student.first_name %>,

-

- Automated assessment did not pass for the tasks below. Please review the Overseer report in <%= @doubtfire_product_name %>, verify your output, and resubmit. -

-

- Tasks to review: -

-

-

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> -

-
- - - +<% content_for :icon do %>alert<% end %> +<% content_for :preheader do %>Automated assessment did not pass for some of your tasks.<% end %> +<% content_for :heading do %>Automated feedback needs your attention<% end %> +<% content_for :subheading do %><%= @tasks.count %> task<%= "s" unless @tasks.count == 1 %> to review<% end %> + +

Hi <%= @student.first_name %>,

+<%= render 'layouts/notification_mail/details', body: "Automated assessment did not pass for the task#{"s" unless @tasks.count == 1} below. Check the Overseer report, verify your output, then resubmit.", category: 'alert', label: 'What to do' %> + +

Tasks to review

+<%= render 'layouts/notification_mail/task_rows', tasks: @tasks, project: @project, host: @doubtfire_host %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/dashboard", label: "Open your tasks in #{@doubtfire_product_name}" %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/portfolio_evidence_mailer/overseer_assessment_failed.text.erb b/app/views/portfolio_evidence_mailer/overseer_assessment_failed.text.erb index 37248debfa..ee7634c353 100644 --- a/app/views/portfolio_evidence_mailer/overseer_assessment_failed.text.erb +++ b/app/views/portfolio_evidence_mailer/overseer_assessment_failed.text.erb @@ -1,12 +1,17 @@ Hi <%= @student.first_name %>, -Automated assessment did not pass for the tasks below. Please review the Overseer report in <%= @doubtfire_product_name %>, verify your output, and resubmit. +WHAT TO DO +Automated assessment did not pass for the task<%= "s" unless @tasks.count == 1 %> below. Check the Overseer report in <%= @doubtfire_product_name %>, verify your output, then resubmit. -Tasks to review: +TASKS TO REVIEW <% @tasks.each do |task| %> * <%= task.task_definition.abbreviation %> - <%= task.task_definition.name %> + <%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard/<%= ERB::Util.url_encode(task.task_definition.abbreviation) %> <% end %> +Open your tasks in <%= @doubtfire_product_name %>: +<%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard + Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> diff --git a/app/views/portfolio_evidence_mailer/portfolio_failed.html.erb b/app/views/portfolio_evidence_mailer/portfolio_failed.html.erb index 8857c89996..d4867ea2e1 100644 --- a/app/views/portfolio_evidence_mailer/portfolio_failed.html.erb +++ b/app/views/portfolio_evidence_mailer/portfolio_failed.html.erb @@ -1,64 +1,23 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

-

Hi <%= @student.first_name %>,

- <% if @project.portfolio_auto_generated %> -

- Profile generation has been automatically triggered. You can customise your portfolio and recreate up until the due date indicated by the teaching staff. -

- <% end %> -

- There was an error compiling your portfolio. This usually means that one of your task PDFs is corrupt. Login to <%= @doubtfire_product_name %> and check each of your task PDFs. Once you have checked that all of the PDFs are valid you can schedule your portfolio to compile again. -

-

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %> -

-
- - - +<% content_for :icon do %>alert<% end %> +<% content_for :preheader do %>There was an error compiling your portfolio.<% end %> +<% content_for :heading do %>Your portfolio could not be compiled<% end %> + +

Hi <%= @student.first_name %>,

+<%= render 'layouts/notification_mail/details', body: "One of your task PDFs is usually the cause. Open each task PDF and check it, then schedule your portfolio to compile again.", category: 'alert', label: 'What to do' %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/portfolio", label: 'Open your portfolio' %> + +<% if @project.portfolio_auto_generated %> + <%= render 'layouts/notification_mail/note', body: "This portfolio was generated for you automatically. You can customise it and recreate it up until the due date your teaching staff set." %> +<% end %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/portfolio_evidence_mailer/portfolio_failed.text.erb b/app/views/portfolio_evidence_mailer/portfolio_failed.text.erb index d43df5b678..f2d247f4b5 100644 --- a/app/views/portfolio_evidence_mailer/portfolio_failed.text.erb +++ b/app/views/portfolio_evidence_mailer/portfolio_failed.text.erb @@ -1,9 +1,12 @@ Hi <%= @student.first_name %>, -<% if @project.portfolio_auto_generated %>Profile generation has been automatically triggered. You can customise your portfolio and recreate up until the due date indicated by the teaching staff.<% end %> - -There was an error compiling your portfolio. This usually means that one of your task PDFs is corrupt. Login to <%= @doubtfire_product_name %> and check each of your task PDFs. Once you have checked that all of the PDFs are valid you can schedule your portfolio to compile again. +WHAT TO DO +There was an error compiling your portfolio. One of your task PDFs is usually the cause. Open each task PDF and check it, then schedule your portfolio to compile again. +<%= @doubtfire_host %>/projects/<%= @project.id %>/portfolio +<% if @project.portfolio_auto_generated %> +This portfolio was generated for you automatically. You can customise it and recreate it up until the due date your teaching staff set. +<% end %> Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %> diff --git a/app/views/portfolio_evidence_mailer/portfolio_ready.html.erb b/app/views/portfolio_evidence_mailer/portfolio_ready.html.erb index fa52bd5a0a..d860cc4333 100644 --- a/app/views/portfolio_evidence_mailer/portfolio_ready.html.erb +++ b/app/views/portfolio_evidence_mailer/portfolio_ready.html.erb @@ -1,66 +1,23 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

-

Hi <%= @student.first_name %>,

- <% if @project.portfolio_auto_generated %> -

- Profile generation has been automatically triggered. You can customise your portfolio and recreate up until the due date indicated by the teaching staff. -

- <% end %> -

- Your portfolio has been compiled and can now be viewed and downloaded from <%= @doubtfire_product_name %>. -

-

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %> -

-
- - - +<% content_for :icon do %>portfolio<% end %> +<% content_for :preheader do %>Your portfolio has been compiled and is ready to view.<% end %> +<% content_for :heading do %>Your portfolio is ready<% end %> +

Hi <%= @student.first_name %>,

+<%= render 'layouts/notification_mail/details', body: "Your portfolio has been compiled. You can view it and download it now.", category: 'portfolio', label: 'Ready' %> +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/portfolio", label: 'Open your portfolio' %> + +<% if @project.portfolio_auto_generated %> + <%= render 'layouts/notification_mail/note', body: "This portfolio was generated for you automatically. You can customise it and recreate it up until the due date your teaching staff set." %> +<% end %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/portfolio_evidence_mailer/portfolio_ready.text.erb b/app/views/portfolio_evidence_mailer/portfolio_ready.text.erb index 1f42cd016c..844562f799 100644 --- a/app/views/portfolio_evidence_mailer/portfolio_ready.text.erb +++ b/app/views/portfolio_evidence_mailer/portfolio_ready.text.erb @@ -1,9 +1,12 @@ Hi <%= @student.first_name %>, -<% if @project.portfolio_auto_generated %>Profile generation has been automatically triggered. You can customise your portfolio and recreate up until the due date indicated by the teaching staff.<% end %> - -Your portfolio has been compiled and can now be viewed and downloaded from <%= @doubtfire_product_name %>. +READY +Your portfolio has been compiled. You can view it and download it now. +<%= @doubtfire_host %>/projects/<%= @project.id %>/portfolio +<% if @project.portfolio_auto_generated %> +This portfolio was generated for you automatically. You can customise it and recreate it up until the due date your teaching staff set. +<% end %> Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @convenor.name %> diff --git a/app/views/portfolio_evidence_mailer/task_feedback_ready.html.erb b/app/views/portfolio_evidence_mailer/task_feedback_ready.html.erb index 362e6f86b6..32f18b1ea4 100644 --- a/app/views/portfolio_evidence_mailer/task_feedback_ready.html.erb +++ b/app/views/portfolio_evidence_mailer/task_feedback_ready.html.erb @@ -1,70 +1,27 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

-

Hi <%= @student.first_name %>,

-

- I have checked your tasks and provided some feedback. Login and check the status <%= @hasComments ? 'and comments' : ''%> of the following tasks: -

-

-

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> -

-
- - - +<% content_for :icon do %>feedback<% end %> +<% content_for :preheader do %>Your tutor has checked your tasks and provided feedback.<% end %> +<% content_for :heading do %>Feedback ready to review<% end %> +<% content_for :subheading do %><%= @tasks.count %> task<%= "s" unless @tasks.count == 1 %> checked by <%= @tutor.name %><% end %> + +

Hi <%= @student.first_name %>,

+<%= render 'layouts/notification_mail/details', body: "I have checked your tasks and left feedback. Open each one to see its new status#{@has_comments ? ' and read the comments' : ''}.", category: 'feedback', label: 'What to do' %> + +

Tasks checked

+<%= render 'layouts/notification_mail/task_rows', + tasks: @tasks, + project: @project, + host: @doubtfire_host, + note_for: ->(task) { task.is_last_comment_by?(@tutor) ? '- with comments' : nil } %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/dashboard", label: "Open your tasks in #{@doubtfire_product_name}" %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/portfolio_evidence_mailer/task_feedback_ready.text.erb b/app/views/portfolio_evidence_mailer/task_feedback_ready.text.erb index d40a8c7ce5..d15221346b 100644 --- a/app/views/portfolio_evidence_mailer/task_feedback_ready.text.erb +++ b/app/views/portfolio_evidence_mailer/task_feedback_ready.text.erb @@ -1,11 +1,17 @@ Hi <%= @student.first_name %>, -I have checked your tasks and provided some feedback. Login and check the status<%= @has_comments ? ' and comments' : ''%> of the following tasks: +WHAT TO DO +I have checked your tasks and left feedback. Open each one to see its new status<%= @has_comments ? ' and read the comments' : ''%>. +TASKS CHECKED <% @tasks.each do |task| %> * <%= task.task_definition.abbreviation %> - <%=task.task_definition.name%> <%= task.is_last_comment_by?(@tutor) ? ' - with comments' : ''%> + <%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard/<%= ERB::Util.url_encode(task.task_definition.abbreviation) %> <% end %> +Open your tasks in <%= @doubtfire_product_name %>: +<%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard + Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> diff --git a/app/views/portfolio_evidence_mailer/task_pdf_failed.html.erb b/app/views/portfolio_evidence_mailer/task_pdf_failed.html.erb index 99d27c61e1..b827a5d31a 100644 --- a/app/views/portfolio_evidence_mailer/task_pdf_failed.html.erb +++ b/app/views/portfolio_evidence_mailer/task_pdf_failed.html.erb @@ -1,72 +1,23 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

-

Hi <%= @student.first_name %>,

-

- Something went wrong with the processing of the following tasks. Please check what you have submitted and try uploading them again. Watch out for character encodings, and file formats. -

-

- The tasks with issues are: -

-

-

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> -

-
- - - +<% content_for :icon do %>alert<% end %> +<% content_for :preheader do %>Something went wrong with the processing of your submitted tasks.<% end %> +<% content_for :heading do %>Task submission processing failed<% end %> +<% content_for :subheading do %><%= @tasks.count %> task<%= "s" unless @tasks.count == 1 %> could not be processed<% end %> + +

Hi <%= @student.first_name %>,

+<%= render 'layouts/notification_mail/details', body: "Check what you submitted and upload it again. Character encodings and file formats are the usual cause.", category: 'alert', label: 'What to do' %> + +

Tasks with issues

+<%= render 'layouts/notification_mail/task_rows', tasks: @tasks, project: @project, host: @doubtfire_host %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/dashboard", label: "Open your tasks in #{@doubtfire_product_name}" %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/portfolio_evidence_mailer/task_pdf_failed.text.erb b/app/views/portfolio_evidence_mailer/task_pdf_failed.text.erb index 04e33ce025..134c7caab3 100644 --- a/app/views/portfolio_evidence_mailer/task_pdf_failed.text.erb +++ b/app/views/portfolio_evidence_mailer/task_pdf_failed.text.erb @@ -1,12 +1,17 @@ Hi <%= @student.first_name %>, -Something went wrong with the processing of the following tasks. Please check what you have submitted and try uploading them again. Watch out for character encodings, and file formats. +WHAT TO DO +Check what you submitted and upload it again. Character encodings and file formats are the usual cause. -The tasks with issues are: +TASKS WITH ISSUES <% @tasks.each do |task| %> * <%= task.task_definition.abbreviation %> - <%=task.task_definition.name%> + <%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard/<%= ERB::Util.url_encode(task.task_definition.abbreviation) %> <% end %> +Open your tasks in <%= @doubtfire_product_name %>: +<%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard + Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> @@ -14,4 +19,4 @@ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> Visit <%= @unsubscribe_url%> to unsubscribe from these notifications. -Generated with <%= @doubtfire_product_name %> \ No newline at end of file +Generated with <%= @doubtfire_product_name %> diff --git a/app/views/portfolio_evidence_mailer/task_pdf_ready_message.html.erb b/app/views/portfolio_evidence_mailer/task_pdf_ready_message.html.erb index 87810ec5f0..8af2613242 100644 --- a/app/views/portfolio_evidence_mailer/task_pdf_ready_message.html.erb +++ b/app/views/portfolio_evidence_mailer/task_pdf_ready_message.html.erb @@ -1,69 +1,23 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

-

Hi <%= @student.first_name %>,

-

- The following evidence you submitted to <%= @doubtfire_product_name %> is now prepared and ready to be viewed and assessed by your tutor: -

-

-

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> -

-
- - - +<% content_for :icon do %>ready_for_feedback<% end %> +<% content_for :preheader do %>Your submitted evidence is ready to be viewed and assessed.<% end %> +<% content_for :heading do %>Task PDFs ready to view<% end %> +<% content_for :subheading do %><%= @tasks.count %> task<%= "s" unless @tasks.count == 1 %> prepared for your tutor<% end %> + +

Hi <%= @student.first_name %>,

+<%= render 'layouts/notification_mail/details', body: "Your evidence has been prepared and is now with your tutor to assess. There is nothing more for you to do on these.", category: 'ready_for_feedback', label: 'Status' %> + +

Ready to view

+<%= render 'layouts/notification_mail/task_rows', tasks: @tasks, project: @project, host: @doubtfire_host %> + +<%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/projects/#{@project.id}/dashboard", label: "Open your tasks in #{@doubtfire_product_name}" %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/portfolio_evidence_mailer/task_pdf_ready_message.text.erb b/app/views/portfolio_evidence_mailer/task_pdf_ready_message.text.erb index f279c6203c..b6c0ebdaf1 100644 --- a/app/views/portfolio_evidence_mailer/task_pdf_ready_message.text.erb +++ b/app/views/portfolio_evidence_mailer/task_pdf_ready_message.text.erb @@ -1,11 +1,17 @@ Hi <%= @student.first_name %>, -The following evidence you submitted to <%= @doubtfire_product_name %> is now prepared and ready to be viewed and assessed by your tutor: +STATUS +Your evidence has been prepared and is now with your tutor to assess. There is nothing more for you to do on these. +READY TO VIEW <% @tasks.each do |task| %> * <%= task.task_definition.abbreviation %> - <%=task.task_definition.name%> + <%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard/<%= ERB::Util.url_encode(task.task_definition.abbreviation) %> <% end %> +Open your tasks in <%= @doubtfire_product_name %>: +<%= @doubtfire_host %>/projects/<%= @project.id %>/dashboard + Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> @@ -13,4 +19,4 @@ The <%= @doubtfire_product_name %> Team on behalf of <%= @tutor.name %> Visit <%= @unsubscribe_url%> to unsubscribe from these notifications. -Generated with <%= @doubtfire_product_name %> \ No newline at end of file +Generated with <%= @doubtfire_product_name %> diff --git a/app/views/tutor_note_mailer/notify_tutor_note.html.erb b/app/views/tutor_note_mailer/notify_tutor_note.html.erb index 74b88d8ff8..e2fc7bfa3d 100644 --- a/app/views/tutor_note_mailer/notify_tutor_note.html.erb +++ b/app/views/tutor_note_mailer/notify_tutor_note.html.erb @@ -1,113 +1,35 @@ - - - - - - -
-

<%= @doubtfire_product_name %> Notification

- -

Hi <%= @recipient.first_name %>,

- -

- You have received a new tutor note from <%= @from.name %> in <%= @unit.name %>. -

- - <% if @recipient == @unit.unit_role_for(@from)&.mentor&.user %> -

You are receiving this message because you are their mentor.

- <% else %> -

You are receiving this message because <%= @from.name %> is your mentor.

- <% end %> - -
- From <%= @from.name %>
- <%= simple_format(@note.note) %> -
- - <% if @task.present? %> -
- This note relates to feedback for a task.

- - Task: - <%= @task.task_definition.abbreviation %> <%= @task.task_definition.name %>
- - Student: - <%= @task.project.user.name %>

- - - View this task - -
- <% end %> - -

- You can view and respond to this note in - <%= @doubtfire_product_name %>. -

- -

- Cheers,
- The <%= @doubtfire_product_name %> Team on behalf of <%= @from.name %> -

- -
- - - - +<% content_for :icon do %>comment<% end %> +<% content_for :preheader do %>A new tutor note from <%= @from.name %> in <%= @unit.name %>.<% end %> +<% content_for :heading do %>New tutor note<% end %> +<% content_for :subheading do %><%= @unit.name %><% end %> + +

Hi <%= @recipient.first_name %>,

+ +<%= render 'layouts/notification_mail/details', body: simple_format(@note.note, style: 'margin:0 0 8px 0;'), category: 'comment', label: "From #{@from.name}" %> + +<% if @task.present? %> + <%= render 'layouts/notification_mail/facts', rows: [ + ['Task', "#{@task.task_definition.abbreviation} - #{@task.task_definition.name}"], + ['Student', @task.project.user.name] + ] %> + <%= render 'layouts/notification_mail/button', url: "#{@doubtfire_host}/units/#{@task.project.unit_id}/tasks/definition/#{@task.project.user.id}/#{ERB::Util.url_encode(@task.task_definition.abbreviation)}", label: 'View the task' %> +<% else %> + <%= render 'layouts/notification_mail/button', url: @doubtfire_host, label: "Open #{@doubtfire_product_name}" %> +<% end %> + +<% if @recipient == @unit.unit_role_for(@from)&.mentor&.user %> + <%= render 'layouts/notification_mail/note', body: "You are receiving this because you are #{@from.name}'s mentor." %> +<% else %> + <%= render 'layouts/notification_mail/note', body: "You are receiving this because #{@from.name} is your mentor." %> +<% end %> + +

+ Cheers,
+ The <%= @doubtfire_product_name %> Team on behalf of <%= @from.name %> +

+ +<% content_for :footer do %> +

+ Unsubscribe | Generated with <%= @doubtfire_product_name %> +

+<% end %> diff --git a/app/views/tutor_note_mailer/notify_tutor_note.text.erb b/app/views/tutor_note_mailer/notify_tutor_note.text.erb index 4a07c824ab..ebdc490f53 100644 --- a/app/views/tutor_note_mailer/notify_tutor_note.text.erb +++ b/app/views/tutor_note_mailer/notify_tutor_note.text.erb @@ -1,30 +1,28 @@ Hi <%= @recipient.first_name %>, -You have received a new tutor note from <%= @from.name %> in <%= @unit.name %>. - -<% if @recipient == @unit.unit_role_for(@from)&.mentor&.user %> -You are receiving this message because you are their mentor. -<% else %> -You are receiving this message because <%= @from.name %> is your mentor. -<% end %> - -Note: +New tutor note in <%= @unit.name %>. +FROM <%= @from.name.upcase %> <%= @note.note %> +<% if @task.present? -%> -<% if @task.present? %> - -This note relates to feedback for a task. +TASK: <%= @task.task_definition.abbreviation %> - <%= @task.task_definition.name %> +STUDENT: <%= @task.project.user.name %> -Task: <%= @task.task_definition.abbreviation %> <%= @task.task_definition.name %> -Student: <%= @task.project.user.name %> +View the task: +<%= @doubtfire_host%>/units/<%=@task.project.unit_id%>/tasks/definition/<%=@task.project.user.id%>/<%= ERB::Util.url_encode(@task.task_definition.abbreviation) %> +<% else -%> -You can view this task here: -<%= @doubtfire_host%>/projects/<%=@task.project.id%>/dashboard/<%=@task.task_definition.abbreviation%>?tutor=true +View and respond to this note in <%= @doubtfire_product_name %>: +<%= @doubtfire_host %> +<% end -%> +<% if @recipient == @unit.unit_role_for(@from)&.mentor&.user -%> -<% end %> +You are receiving this because you are <%= @from.name %>'s mentor. +<% else -%> -You can view and respond to this note in <%= @doubtfire_product_name %>. +You are receiving this because <%= @from.name %> is your mentor. +<% end -%> Cheers, The <%= @doubtfire_product_name %> Team on behalf of <%= @from.name %> diff --git a/config/environments/development.rb b/config/environments/development.rb index 05d01df74c..f87cbef55d 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -60,9 +60,28 @@ config.action_mailer.perform_caching = false - # Tell Action Mailer not to deliver emails to the real world. - # Write them to file instead (under doubtfire-api/tmp/mails) - config.action_mailer.delivery_method = :file + # Never deliver email to the real world in development. + # + # With docker (the normal case), DF_SMTP_ADDRESS points at the mailpit + # container and every email shows up in a web inbox at http://localhost:8025. + # Mailpit accepts everything and forwards nothing. + # + # Without it, mail is written to a file instead. Under docker that file lands + # on the host at doubtfire-deploy/data/tmp/mails/, NOT in this repository, + # because development/docker-compose.yml mounts ../data/tmp over /doubtfire/tmp + # and Rails.root in the container is /doubtfire. Looking for it under + # doubtfire-api/tmp/mails shows an empty folder and makes email look broken. + # + # See doubtfire-deploy/RUNNING-LOCALLY.md. + if ENV['DF_SMTP_ADDRESS'].present? + config.action_mailer.delivery_method = :smtp + config.action_mailer.smtp_settings = { + address: ENV['DF_SMTP_ADDRESS'], + port: ENV.fetch('DF_SMTP_PORT', 1025).to_i + } + else + config.action_mailer.delivery_method = :file + end # Print deprecation notices to the Rails logger. config.active_support.deprecation = :log diff --git a/config/initializers/sidekiq.rb b/config/initializers/sidekiq.rb index 942c043b5d..9c571cfb91 100644 --- a/config/initializers/sidekiq.rb +++ b/config/initializers/sidekiq.rb @@ -17,7 +17,23 @@ config.on(:startup) do schedule_file = Rails.root.join('config/schedule.yml') - Sidekiq::Cron::Job.load_from_hash!(YAML.load_file(schedule_file)) if File.exist?(schedule_file) + + # 'source' => 'schedule' is what makes the bang in load_from_hash! do + # anything. It calls destroy_removed_jobs, which only ever considers jobs + # whose source is "schedule", and Sidekiq::Cron::Job defaults every other + # job to "dynamic". Loading without it registered all of these as dynamic, + # so a job deleted or renamed in this file was never removed from Redis: it + # kept its schedule and kept firing, and after a rename both the old name + # and the new one ran. The key has to be a string, because the job reads + # args["source"] out of a hash that YAML filled with string keys. + # + # Jobs added by hand in the Sidekiq web UI stay dynamic and are still left + # alone, which is the point of the distinction. Entries already in Redis + # from before this change are also still dynamic, so each one has to be + # destroyed once by hand or it will outlive its line in this file. + if File.exist?(schedule_file) + Sidekiq::Cron::Job.load_from_hash!(YAML.load_file(schedule_file), 'source' => 'schedule') + end end end diff --git a/config/initializers/vapid_configuration.rb b/config/initializers/vapid_configuration.rb new file mode 100644 index 0000000000..bcfc559f3b --- /dev/null +++ b/config/initializers/vapid_configuration.rb @@ -0,0 +1,6 @@ +# frozen_string_literal: true + +require_relative '../../lib/vapid_configuration' + +VapidConfiguration.validate!(production: Rails.env.production?) +Rails.application.config.filter_parameters += %i[doubtfire_vapid_private_key vapid_private_key private_key] diff --git a/config/institution.yml b/config/institution.yml index 74d7d7c8b0..c8324c9892 100644 --- a/config/institution.yml +++ b/config/institution.yml @@ -1,5 +1,6 @@ name: Doubtfire University email_domain: doubtfire.com +email_sender: noreply@doubtfire.local host: localhost:3000 cookie_domain: localhost product_name: Doubtfire diff --git a/config/schedule.yml b/config/schedule.yml index 62fd893daf..eac9abcf17 100644 --- a/config/schedule.yml +++ b/config/schedule.yml @@ -1,4 +1,8 @@ # config/schedule.yml +# +# Sidekiq-cron owns the in-process scheduled jobs listed below. +# The container's rake schedule lives in .ci-setup/crontab. +# Edit this file for a Ruby job, .ci-setup/crontab for a rake task. register_webhooks: cron: "every day at 5" @@ -16,14 +20,111 @@ refresh_moderation_feedback_timestamps: cron: "every 60 minutes" class: "RefreshModerationFeedbackTimestampsJob" +aggregate_peer_progress: + cron: "every day at 11:45pm" + class: "AggregatePeerProgressJob" + aggregate_task_completion_stats: cron: "every day at 11:55pm" class: "AggregateTaskCompletionStatsJob" +# Nightly plagiarism scan. This is the run that used to live only in the +# container crontab, moved here so it is visible next to the other recurring +# jobs. The same scan can now be triggered on demand from the unit, which is +# why it belongs in Sidekiq rather than cron. +check_unit_similarity: + cron: "every day at 5" + class: "CheckUnitSimilarityJob" + poll_communication_set_schedules: cron: "every 5 minutes" class: "PollCommunicationSetSchedulesJob" +# Recheck effective task start dates once a day. Directly created, copied and +# imported tasks are queued immediately; this sweep handles students whose +# flexible or future-dated start arrives later, with a bounded catch-up window +# for short worker outages and late enrolment. +send_new_task_available_notifications: + cron: "every day at 8:10am" + class: "SendNewTaskAvailableNotificationsJob" + +# Once a day, in the morning. +# +# A deadline moves once a day, so nothing is gained by sweeping for one every +# half hour, and every notification this job raises sends an email. Pinning it +# to a time also pins when the emails arrive: on a shorter interval a student +# gets theirs at whatever hour the task happened to cross into the window, +# which is 3am about as often as any other hour. +# +# Missing a run costs nothing. The job only asks whether a task is due within +# the next three days, so the following morning still catches everything the +# skipped run would have. +# +# 8am is 8am wherever the Sidekiq process thinks it is. There is no timezone in +# this file and no config.time_zone in the app, so the hour comes from TZ in the +# process environment. development/api.env sets TZ=Australia/Melbourne, which is +# what makes this a morning; a deployment that leaves TZ unset gets the image +# default of UTC and sends these in the evening. The entry above it, at 11:55pm, +# depends on the same thing. +send_due_soon_reminders: + cron: "every day at 8am" + class: "SendDueSoonRemindersJob" + # archive_old_units: # cron: "every 6 months" # class: "ArchiveOldUnitsJob" + +# Default off: the worker returns immediately until an administrator configures +# the approved student-visible channels and Graph application credentials. +sync_teams_announcements: + cron: "every 5 minutes" + class: "SyncTeamsAnnouncementsJob" + +# Unit Hub session reminders for people who opted in. Every five minutes because +# the reminder is 30 minutes before a start time, so a daily sweep cannot land +# it. Each occurrence is reminded once per person however many runs see it. +send_unit_session_reminders: + cron: "every 5 minutes" + class: "SendUnitSessionRemindersJob" + +# The student digest, once per cadence a student can choose. +# +# These three replace `0 7 * * 1 send_weekly_emails.sh` in .ci-setup/crontab, +# which is deleted. That line ran the older per-unit mail, which is still there +# and still works but is no longer on any schedule, because a student on both +# would get the digest and a mail per unit in the same morning. +# +# Scheduled here and not in that crontab because the crontab is only installed +# inside the pdfgen container, so a deployment without LaTeX has never sent a +# summary at all. CheckUnitSimilarityJob above moved for the same reason. +# +# All three at 7am, which is the hour the weekly one has always used, so a +# student who switches cadence sees the mail arrive at the same time of day. It +# is early enough to be waiting when they start and late enough not to buzz a +# phone overnight, which is the whole reason this is pinned to an hour instead +# of running on an interval. Five minutes apart so that on a Monday the 1st the +# three do not all land on the mailers queue at once, and so a log line says +# which one ran. +# +# The hour is 7am wherever the Sidekiq process thinks it is, the same as +# send_due_soon_reminders above. There is no timezone in this file and no +# config.time_zone in the app, so TZ in the process environment decides. +# development/api.env sets TZ=Australia/Melbourne; a deployment that leaves TZ +# unset gets the image default of UTC and mails these in the evening. +# +# The 1st for the monthly run because it is the one day of the month every month +# has, and a summary that arrives on the 1st covers the month that just ended. +send_daily_digest_emails: + cron: "every day at 7am" + class: "SendDigestEmailsJob" + args: ["daily"] + +send_weekly_digest_emails: + cron: "every monday at 7:05am" + class: "SendDigestEmailsJob" + args: ["weekly"] + +send_monthly_digest_emails: + cron: "every month on the 1st at 7:10am" + class: "SendDigestEmailsJob" + args: ["monthly"] diff --git a/config/sidekiq.yml b/config/sidekiq.yml index 0515ae2186..9b84a6d2b4 100644 --- a/config/sidekiq.yml +++ b/config/sidekiq.yml @@ -1 +1,27 @@ -:concurrency: 1 +# Keep the historical single-worker default for small installations. Production +# deployments can raise this after sizing the database pool and worker memory. +<% sidekiq_concurrency = Integer(ENV.fetch('DF_SIDEKIQ_CONCURRENCY', '1'), 10) %> +<% raise ArgumentError, 'DF_SIDEKIQ_CONCURRENCY must be positive' unless sidekiq_concurrency.positive? %> +:concurrency: <%= sidekiq_concurrency %> + +# Strict priority, highest first. User-facing notification email and Web Push +# use `mailers` and `notifications` respectively. Each job performs one channel +# delivery, so neither must wait behind AcceptSubmissionJob or a CSV export on +# `default` at concurrency 1. +# +# This list is what makes both notification channel jobs run in production. +# That worker is started by lib/shell/sidekiq_entry_point.sh, a bare +# `bundle exec sidekiq` with no -q, so with no :queues: here Sidekiq would listen +# on `default` alone and notification channel jobs would sit in Redis unread. +# perform_async succeeds, nothing raises and nothing logs, so the failure is +# silent. +# +# The development worker passes `-q mailers -q notifications` on the command +# line, which replaces this list, so it stays narrow and keeps ignoring +# `default`. See the comment on the doubtfire-sidekiq service in doubtfire-deploy +# development/docker-compose.yml. +:queues: + - mailers + - notifications + - submissions + - default diff --git a/db/migrate/20260722000001_create_notifications.rb b/db/migrate/20260722000001_create_notifications.rb new file mode 100644 index 0000000000..f935c74ed8 --- /dev/null +++ b/db/migrate/20260722000001_create_notifications.rb @@ -0,0 +1,15 @@ +class CreateNotifications < ActiveRecord::Migration[8.0] + def change + create_table :notifications do |t| + t.references :user, foreign_key: true, null: false + t.string :notification_type, null: false + t.string :message, null: false + t.string :link + t.datetime :read_at + + t.timestamps + end + + add_index :notifications, [:user_id, :read_at] + end +end diff --git a/db/migrate/20260802000001_add_event_to_notifications.rb b/db/migrate/20260802000001_add_event_to_notifications.rb new file mode 100644 index 0000000000..1209aed464 --- /dev/null +++ b/db/migrate/20260802000001_add_event_to_notifications.rb @@ -0,0 +1,21 @@ +class AddEventToNotifications < ActiveRecord::Migration[8.0] + # `notification_type` is the broad category the user's preferences switch on + # (task, feedback, portfolio, extension, general). `event` records which + # specific thing happened, so a notification can be traced back to the code + # that raised it and so we can later suppress or batch a single event without + # turning off the whole category. + def up + # Added with a placeholder default first, so the ALTER succeeds on a + # database that already has notification rows, then the default is dropped + # so new records must supply a real event. + add_column :notifications, :event, :string, null: false, default: 'legacy' + change_column_default :notifications, :event, nil + + add_index :notifications, [:user_id, :event] + end + + def down + remove_index :notifications, column: [:user_id, :event] + remove_column :notifications, :event + end +end diff --git a/db/migrate/20260802000002_change_notification_message_to_text.rb b/db/migrate/20260802000002_change_notification_message_to_text.rb new file mode 100644 index 0000000000..2a5bf739d1 --- /dev/null +++ b/db/migrate/20260802000002_change_notification_message_to_text.rb @@ -0,0 +1,17 @@ +class ChangeNotificationMessageToText < ActiveRecord::Migration[8.0] + # The model allows a 500 character message, but the column was created as a + # string, which MariaDB stores as VARCHAR(255). Anything over 255 characters + # passed validation and then failed at the database. TEXT holds the full + # validated length. + def up + change_column :notifications, :message, :text, null: false + end + + # Lossy. Narrowing back to VARCHAR(255) will reject (strict mode) or silently + # truncate (permissive mode) any message over 255 characters saved while the + # column was text. Only roll this back on a database you are willing to lose + # long messages from. + def down + change_column :notifications, :message, :string, null: false + end +end diff --git a/db/migrate/20260802000003_create_push_subscriptions.rb b/db/migrate/20260802000003_create_push_subscriptions.rb new file mode 100644 index 0000000000..8d444927ec --- /dev/null +++ b/db/migrate/20260802000003_create_push_subscriptions.rb @@ -0,0 +1,28 @@ +class CreatePushSubscriptions < ActiveRecord::Migration[8.0] + def change + create_table :push_subscriptions do |t| + t.references :user, foreign_key: true, null: false + + # The push service URL the browser hands us. We send to it, and it + # identifies the browser rather than the person, so it is unique across + # the whole table and not just per user. + # + # 500 rather than the default 255 because Firefox and Safari endpoints + # run close to 260 characters. A varchar(255) would reject them under + # strict mode and silently truncate them otherwise, and a truncated + # endpoint is a push that quietly goes nowhere. utf8mb4 makes this a + # 2000 byte index key, inside InnoDB's 3072 byte limit. + t.string :endpoint, null: false, limit: 500 + + # The browser's public key and auth secret. The payload is encrypted to + # these, so without them a push cannot be read by the browser that asked + # for it. + t.string :p256dh, null: false + t.string :auth, null: false + + t.timestamps + end + + add_index :push_subscriptions, :endpoint, unique: true + end +end diff --git a/db/migrate/20260824000001_track_new_task_notifications.rb b/db/migrate/20260824000001_track_new_task_notifications.rb new file mode 100644 index 0000000000..b89fcbb8ba --- /dev/null +++ b/db/migrate/20260824000001_track_new_task_notifications.rb @@ -0,0 +1,50 @@ +class TrackNewTaskNotifications < ActiveRecord::Migration[8.0] + def up + # The database default covers definitions written by an older application + # instance during a rolling deployment. Supported workflows replace it + # with their actual tracking boundary once configuration is complete. + unless column_exists?(:task_definitions, :new_task_notifications_from) + add_column( + :task_definitions, + :new_task_notifications_from, + :datetime, + default: -> { 'UTC_TIMESTAMP()' } + ) + end + + unless index_exists?(:task_definitions, :new_task_notifications_from) + add_index :task_definitions, :new_task_notifications_from + end + + unless column_exists?(:notifications, :dedupe_key) + add_column :notifications, :dedupe_key, :string, limit: 191 + end + unless column_exists?(:notifications, :delivered_at) + add_column :notifications, :delivered_at, :datetime + end + + unless index_exists?(:notifications, [:user_id, :dedupe_key], unique: true) + add_index( + :notifications, + [:user_id, :dedupe_key], + unique: true, + name: 'index_notifications_on_user_and_dedupe_key' + ) + end + end + + def down + if index_exists?(:notifications, [:user_id, :dedupe_key], name: 'index_notifications_on_user_and_dedupe_key') + remove_index :notifications, name: 'index_notifications_on_user_and_dedupe_key' + end + remove_column :notifications, :delivered_at if column_exists?(:notifications, :delivered_at) + remove_column :notifications, :dedupe_key if column_exists?(:notifications, :dedupe_key) + + if index_exists?(:task_definitions, :new_task_notifications_from) + remove_index :task_definitions, :new_task_notifications_from + end + if column_exists?(:task_definitions, :new_task_notifications_from) + remove_column :task_definitions, :new_task_notifications_from + end + end +end diff --git a/db/migrate/20260827000001_add_notifiable_to_notifications.rb b/db/migrate/20260827000001_add_notifiable_to_notifications.rb new file mode 100644 index 0000000000..8c56d1ce0e --- /dev/null +++ b/db/migrate/20260827000001_add_notifiable_to_notifications.rb @@ -0,0 +1,24 @@ +class AddNotifiableToNotifications < ActiveRecord::Migration[8.0] + # A notification records what happened (`event`) but never what it happened + # to, so the only tie back to the source is the `link` string. Two comments on + # the same task produce the identical link, so nothing can be joined on it. + # + # `notifiable` is polymorphic rather than a `task_id` because the targets + # already on the board are not all tasks: portfolio submission and extension + # events point at other records. + # + # Nullable, and no backfill. A `general` notification legitimately has no + # target, and there is no honest way to work out what an existing row referred + # to. A guess written into the database is worse than a null. + def up + # The default single column index on `notifiable_type` is useless here, + # every lookup carries both halves, so the compound index is added by hand. + add_reference :notifications, :notifiable, polymorphic: true, null: true, index: false + add_index :notifications, [:notifiable_type, :notifiable_id] + end + + def down + remove_index :notifications, column: [:notifiable_type, :notifiable_id] + remove_reference :notifications, :notifiable, polymorphic: true + end +end diff --git a/db/migrate/20260831012000_create_additional_notification_emails.rb b/db/migrate/20260831012000_create_additional_notification_emails.rb new file mode 100644 index 0000000000..d6b6b5b914 --- /dev/null +++ b/db/migrate/20260831012000_create_additional_notification_emails.rb @@ -0,0 +1,31 @@ +class CreateAdditionalNotificationEmails < ActiveRecord::Migration[8.0] + # Pin the project's standard table options so these tables match every + # other table even when the database default collation differs. Current + # MariaDB images, including the 12.3 one CI uses, default utf8mb4 to + # utf8mb4_uca1400_ai_ci. + TABLE_OPTIONS = 'ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci' + + def change + create_table :additional_notification_emails, options: TABLE_OPTIONS do |t| + t.references :user, null: false, foreign_key: true, index: { unique: true } + t.string :email, null: false, limit: 254 + t.integer :verification_version, null: false, default: 0 + t.datetime :verification_sent_at + t.datetime :verification_expires_at + t.datetime :verified_at + + t.timestamps + end + + create_table :additional_notification_email_audits, options: TABLE_OPTIONS do |t| + t.references :user, null: false, foreign_key: true + t.string :event, null: false, limit: 64 + + t.timestamps + end + + add_index :additional_notification_email_audits, + %i[user_id event created_at], + name: 'idx_additional_email_audits_user_event_time' + end +end diff --git a/db/migrate/20260920010000_add_notification_email_delivery_state.rb b/db/migrate/20260920010000_add_notification_email_delivery_state.rb new file mode 100644 index 0000000000..228d1631d7 --- /dev/null +++ b/db/migrate/20260920010000_add_notification_email_delivery_state.rb @@ -0,0 +1,14 @@ +# frozen_string_literal: true + +class AddNotificationEmailDeliveryState < ActiveRecord::Migration[8.0] + def change + # Historical rows have no reliable delivery evidence. Do not call them sent + # or queue them again when the migration runs. + add_column :notifications, :email_delivery_state, :string, default: 'untracked', null: false + add_column :notifications, :email_delivery_attempts, :integer, default: 0, null: false + add_column :notifications, :email_delivered_at, :datetime + add_column :notifications, :email_delivery_error_class, :string + add_index :notifications, :email_delivery_state + add_index :notifications, [:user_id, :created_at], name: 'index_notifications_on_recipient_rate_window' + end +end diff --git a/db/migrate/20260927000001_add_unit_hub_notification_preferences_to_users.rb b/db/migrate/20260927000001_add_unit_hub_notification_preferences_to_users.rb new file mode 100644 index 0000000000..070b1caa29 --- /dev/null +++ b/db/migrate/20260927000001_add_unit_hub_notification_preferences_to_users.rb @@ -0,0 +1,16 @@ +# frozen_string_literal: true + +# Unit Hub updates get a category of their own, stored as user columns like the +# task, feedback and portfolio categories. Unlike those, email and push are +# separate opt-ins here because an announcement can go to a whole cohort, so +# only the in-app bell is on by default. Session reminders are a fourth opt-in. +class AddUnitHubNotificationPreferencesToUsers < ActiveRecord::Migration[8.0] + def change + change_table :users, bulk: true do |t| + t.boolean :receive_unit_hub_notifications, default: true, null: false + t.boolean :receive_unit_hub_email_notifications, default: false, null: false + t.boolean :receive_unit_hub_push_notifications, default: false, null: false + t.boolean :receive_unit_hub_session_reminders, default: false, null: false + end + end +end diff --git a/db/migrate/20260927000002_add_digest_frequency_to_users.rb b/db/migrate/20260927000002_add_digest_frequency_to_users.rb new file mode 100644 index 0000000000..d65000c54f --- /dev/null +++ b/db/migrate/20260927000002_add_digest_frequency_to_users.rb @@ -0,0 +1,13 @@ +# frozen_string_literal: true + +# How often a student wants the unit summary email. Until now that email went out +# weekly to everyone with feedback notifications on, with no way to ask for it +# more or less often, and no way to stop it without also losing feedback mail. +# +# 'weekly' is the default because it is what every existing account already +# receives, so adding the column changes nothing until someone chooses. +class AddDigestFrequencyToUsers < ActiveRecord::Migration[8.0] + def change + add_column :users, :digest_frequency, :string, default: 'weekly', null: false + end +end diff --git a/docs/email-notification-first-pass-tests.md b/docs/email-notification-first-pass-tests.md new file mode 100644 index 0000000000..92de9c4055 --- /dev/null +++ b/docs/email-notification-first-pass-tests.md @@ -0,0 +1,29 @@ +# Email Notification – First Pass Test Cases + +## Purpose + +The purpose of this document is to define an initial set of test cases for user email notification preferences and correct email delivery before implementation begins. + +This task documents expected behaviour only. No production code has been modified. + +## Test Cases + +| Test ID | Scenario | Preconditions | Test Action | Expected Result | +|---|---|---|---|---| +| EN-01 | Notifications enabled | The user has enabled email notifications and has a valid email address | Trigger a valid notification event | Exactly one email is delivered to the intended recipient | +| EN-02 | Notifications disabled | The user has disabled email notifications | Trigger the same notification event | No email is created, queued, or delivered | +| EN-03 | Wrong recipient | The event belongs to User A, while User B also exists in the system | Trigger the notification for User A | Only User A receives the email; User B receives nothing | +| EN-04 | Duplicate event | The same notification event is processed twice | Process the duplicate event | Only one email is delivered | +| EN-05 | Changed preference | The user changes the preference from enabled to disabled before the event | Trigger a notification after the preference change | The updated preference is respected and no email is delivered | + +## Key Finding + +Correct email delivery depends on validating both the user's latest notification preference and the intended recipient before sending the email. + +## Recommended Next Step + +Confirm how duplicate notification events will be identified and determine where automated tests for these scenarios should be implemented when the feature is developed. + +## Current Blocker + +The email notification feature is not yet fully implemented, so these test cases define expected behaviour only and cannot yet be executed as automated tests. \ No newline at end of file diff --git a/docs/email_notifications/safe_starter_email_templates.md b/docs/email_notifications/safe_starter_email_templates.md new file mode 100644 index 0000000000..2cad342331 --- /dev/null +++ b/docs/email_notifications/safe_starter_email_templates.md @@ -0,0 +1,173 @@ + +# Safe Starter Email Templates + +## Purpose + +These starter templates provide privacy-aware wording for OnTrack email +notifications. They avoid exposing unnecessary assessment information in email +subject lines and bodies. + +Recipients should sign in to OnTrack to view task names, unit information, +feedback, marks, comments, submissions, dates, and other assessment details. + +## Privacy Guidelines + +- Keep subject lines generic. +- Do not include student names in subject lines. +- Do not include unit names or task names in subject lines. +- Do not include marks, grades, feedback text, tutor comments, or submission details. +- Direct users to sign in to OnTrack to view protected information. +- Do not include personal information or authentication tokens in URLs. +- Respect the user's notification preferences. +- Use configured OnTrack names and URLs instead of hard-coded deployment details. + +## Suggested Placeholders + +- `{{product_name}}`: the configured system name, such as OnTrack. +- `{{sign_in_url}}`: a secure link to the OnTrack sign-in page. +- `{{notification_settings_url}}`: the user's notification settings page. + +--- + +## 1. Due Soon + +### Subject + +```text +{{product_name}} reminder: a due date is approaching +``` + +### Body + +```text +Hello, + +A task in {{product_name}} is due soon. + +Sign in to review the task and confirm the due date: +{{sign_in_url}} + +No assessment details are included in this email. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### Privacy Note + +This email does not include the task name, unit name, due date, submission +status, or other assessment details. + +--- + +## 2. Feedback Available + +### Subject + +```text +{{product_name}} notification: feedback is available +``` + +### Body + +```text +Hello, + +New feedback is available in {{product_name}}. + +Sign in to view it securely: +{{sign_in_url}} + +No feedback content is included in this email. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### Privacy Note + +This email does not include feedback text, task names, unit names, staff +comments, marks, or assessment results. + +--- + +## 3. Task Marked + +### Subject + +```text +{{product_name}} notification: a task has been marked +``` + +### Body + +```text +Hello, + +A task has been marked in {{product_name}}. + +Sign in to review the outcome and any next steps: +{{sign_in_url}} + +No mark or result is included in this email. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### Privacy Note + +This email does not include the mark, grade, result, task name, unit name, +or feedback. + +--- + +## 4. Date Changed + +### Subject + +```text +{{product_name}} notification: a task date has changed +``` + +### Body + +```text +Hello, + +A date associated with a task has changed in {{product_name}}. + +Sign in to confirm the current date: +{{sign_in_url}} + +Use the date displayed in {{product_name}} as the current source of truth. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### Privacy Note + +This email does not include the task name, unit name, previous date, or new date. + +--- + +## Key Finding + +The API repository already contains notification mailer functionality and +notification-related email views. Future implementation should investigate +reusing the existing mailer structure instead of creating a separate email +delivery system. + +## Recommended Next Step + +Before production implementation, the Email Notifications team should confirm: + +1. The trigger for each notification. +2. The user roles that receive each notification. +3. The secure destination URL for each email. +4. Whether exact dates may be included in email bodies. +5. Whether emails are sent immediately or through a background job. +6. How notification preferences and opt-out behaviour are applied. + +Production mailer code, event triggers, database changes, and frontend notification settings are outside the scope of this starter documentation task. \ No newline at end of file diff --git a/docs/email_notifications/v2_event_email_templates.md b/docs/email_notifications/v2_event_email_templates.md new file mode 100644 index 0000000000..abdc32bf8f --- /dev/null +++ b/docs/email_notifications/v2_event_email_templates.md @@ -0,0 +1,314 @@ +# V2 Event Email Templates (EN-V05 to EN-V08) + +## Purpose + +These templates provide reader-focused email copy for the remaining four v2 +notification events. They follow the privacy rules in +`safe_starter_email_templates.md`: include only the detail needed to explain +what happened, keep assessment content out of email, and direct the recipient +back to OnTrack for the full context. + +The subjects below are the preferred product copy. The notification mailer +currently uses the shared subject `{{product_name}}: New notification` for +every event; adopting event-specific subjects requires a separate mailer +decision. + +## Shared placeholders + +- `{{product_name}}`: the configured product name, such as OnTrack. +- `{{recipient_first_name}}`: the recipient's first name. +- `{{student_name}}`: the student whose task is waiting for marking. Used only + in the tutor-facing EN-V06 email. +- `{{task_name}}`: the task that is waiting for marking. Used only in EN-V06. +- `{{group_name}}`: the group the student joined or left. +- `{{unit_code}}`: the unit code. +- `{{submitted_at}}`: the portfolio receipt date and time in the project's + campus timezone, falling back to the application timezone. Include both the + timezone abbreviation and numeric UTC offset, for example + `AEST (UTC+10:00)`. +- `{{destination_url}}`: an authenticated OnTrack page for the event. +- `{{notification_settings_url}}`: the recipient's notification settings. + +Never substitute marks, grades, feedback, comment text, or portfolio contents +into these placeholders. EN-V06 is the one narrow third-party exception: the +assigned tutor may see the submitting student's name and task name. Do not name +any other student or third party. + +--- + +## EN-V05: Group membership changed + +Two versions are required because joining and leaving are different messages. +Only the affected student receives either version. + +### Added to a group + +#### Subject + +```text +{{product_name}} notification: you joined a group +``` + +#### Plain text body + +```text +Hi {{recipient_first_name}}, + +You have been added to {{group_name}} in {{unit_code}}. + +Open {{product_name}} to view your current group: +{{destination_url}} + +No other group members are named in this email. + +This service notification is always sent. +``` + +#### HTML body + +```html +

Hi {{recipient_first_name}},

+ +

You have been added to {{group_name}} in {{unit_code}}.

+ +

View your current group in {{product_name}}

+ +

No other group members are named in this email.

+ +

This service notification is always sent.

+``` + +### Removed from a group + +#### Subject + +```text +{{product_name}} notification: your group changed +``` + +#### Plain text body + +```text +Hi {{recipient_first_name}}, + +You are no longer a member of {{group_name}} in {{unit_code}}. + +Open {{product_name}} to view your current group information: +{{destination_url}} + +If you did not expect this change, contact your teaching team. + +This service notification is always sent. +``` + +#### HTML body + +```html +

Hi {{recipient_first_name}},

+ +

You are no longer a member of {{group_name}} in {{unit_code}}.

+ +

View your current group information in {{product_name}}

+ +

If you did not expect this change, contact your teaching team.

+ +

This service notification is always sent.

+``` + +### Privacy note + +Neither version names another student or describes why the membership changed. +The removal copy states the current fact without implying fault or punishment. +The event uses the `general` category, which has no preference gate. + +### Implementation alignment + +EN-V05 merged before this copy document. Its current templates render the +notification message's "added to" or "removed from" wording and do not yet +include the destination link, softer removal sentence, or always-sent footer +above. Merging this document does not silently change that event; EN-V05 needs a +focused template-and-test follow-up to adopt the approved copy exactly. + +--- + +## EN-V06: Task submitted for marking + +This is the only tutor-facing email in this document. Naming the student and +task is necessary so the tutor can identify the work waiting for them. Do not +include the submission, a mark, feedback, or a comment. + +### Subject + +```text +{{product_name}} notification: a task is ready for marking +``` + +### Plain text body + +```text +Hi {{recipient_first_name}}, + +{{student_name}} submitted {{task_name}} for marking in {{product_name}}. + +Open the task to review the submission: +{{destination_url}} + +The submission and any assessment content are not included in this email. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### HTML body + +```html +

Hi {{recipient_first_name}},

+ +

{{student_name}} submitted {{task_name}} for marking in {{product_name}}.

+ +

Open the task in {{product_name}}

+ +

The submission and any assessment content are not included in this email.

+ +

+ You can manage your notification preferences in + your profile. +

+``` + +### Privacy note + +The student name and task name are visible only to the assigned tutor. The +subject remains generic and does not identify the student or task. + +--- + +## EN-V07: Portfolio submission received + +This is a receipt for the student who submitted the portfolio. The receipt time +must include an unambiguous timezone. It confirms receipt only; it does not say +that the portfolio is valid, complete, generated, or assessed. + +### Subject + +```text +{{product_name}} receipt: portfolio submission received +``` + +### Plain text body + +```text +Hi {{recipient_first_name}}, + +{{product_name}} received your portfolio submission at {{submitted_at}}. + +Open {{product_name}} to view its current status: +{{destination_url}} + +This receipt confirms when the submission was received. It does not confirm an assessment outcome. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### HTML body + +```html +

Hi {{recipient_first_name}},

+ +

{{product_name}} received your portfolio submission at {{submitted_at}}.

+ +

View the current status in {{product_name}}

+ +

+ This receipt confirms when the submission was received. It does not confirm + an assessment outcome. +

+ +

+ You can manage your notification preferences in + your profile. +

+``` + +### Privacy note + +The email contains the receipt date and time in the project campus timezone +(application timezone fallback), with both an abbreviation and numeric UTC +offset. It contains no portfolio content, marks, feedback, or other student +information. + +--- + +## EN-V08: Discussion or check-in + +### Scope decision required + +OnTrack does not currently have a booking or appointment record that can raise +the event as originally named. `Task#add_discussion_comment` does create an +audio discussion request for a student, so `discussion_request_created` is the +proposed replacement. The copy below is for that replacement and must not be +described as a booking confirmation. The Email Notifications lead must approve +the replacement before its implementation merges. + +If a real booking model is introduced later, write separate copy that includes +the booked date, time, timezone, and location or meeting method from that model. + +### Subject + +```text +{{product_name}} notification: a discussion prompt is ready +``` + +### Plain text body + +```text +Hi {{recipient_first_name}}, + +A discussion prompt is ready for you in {{product_name}}. + +Open the task to listen and respond: +{{destination_url}} + +The prompt is not included in this email. + +You can manage your notification preferences here: +{{notification_settings_url}} +``` + +### HTML body + +```html +

Hi {{recipient_first_name}},

+ +

A discussion prompt is ready for you in {{product_name}}.

+ +

Open the task in {{product_name}}

+ +

The prompt is not included in this email.

+ +

+ You can manage your notification preferences in + your profile. +

+``` + +### Privacy note + +The email does not include audio, prompt content, comments, task or unit names, +or another person's name. The proposed replacement uses the `feedback` +category and respects `receive_feedback_notifications`. + +--- + +## Review checklist + +- Each event has a subject, plain text body, and HTML body. +- EN-V05 covers both joining and leaving a group. +- EN-V06 is written to a tutor and identifies only the student and task needed + to act. +- EN-V07 includes a date, time, timezone abbreviation, and numeric UTC offset + and does not overstate what the receipt proves. +- EN-V08 is not presented as buildable booking functionality. +- No template includes marks, grades, feedback, comment text, portfolio + contents, or unnecessary third-party names. diff --git a/docs/notifications/CONTRIBUTING.md b/docs/notifications/CONTRIBUTING.md new file mode 100644 index 0000000000..0177f78038 --- /dev/null +++ b/docs/notifications/CONTRIBUTING.md @@ -0,0 +1,95 @@ +# Contributing to notifications + +Start with the [API contributor guide](../../CONTRIBUTING.md) for setup, +repositories, branches, commits, tests and review. All three team repositories +use `11.0.x` for integration; open ticket pull requests against that branch. +The former `feature/notifications` branch is historical. Check current code and +existing PRs before implementing an old ticket. The standing branch currency +rule is in [DECISIONS.md](DECISIONS.md). + +For production diagnosis and queue operations, use the [runbook](RUNBOOK.md) +and [delivery operations guide](delivery-operations.md). + +## Channel delivery belongs in Sidekiq + +`NotificationService.notify` persists the in-app record, then queues separate +ID-only email and push jobs. Sidekiq workers reload the notification and perform +provider network I/O; a request only waits for the short Redis hand-offs. Both +jobs avoid the general-purpose `default` queue: email uses `mailers` and Web Push +uses `notifications`. Every deployed environment that should deliver +notifications must run a Sidekiq worker for both channel queues. + +Email is queued by `Notification`'s creation `after_commit` hook, so a job +cannot read an uncommitted notification. A deleted record is a successful no-op +for that worker. `delivered_at` tracks the push queue hand-off, not provider +receipt; email outcomes have separate persisted fields. A failed push hand-off +remains retryable. Delivery is at-least-once: channel jobs take stable IDs, +recheck preferences and tolerate retries. + +Push producers can run inside wider database transactions, so the push worker +must raise when its notification id is not yet visible. Push provider failures +are attempted across all registered browsers and then raised as an aggregate +error so Sidekiq's retry policy is effective. + +**Never loop over a whole cohort and call `NotificationService.notify` directly +from a web request.** Even without provider I/O, that would create one record +and make two queue round trips per recipient before the request can finish. The +current new-task and due-date events avoid that by enqueueing +`NewTaskAvailableNotificationJob` and `TaskDueDateChangedNotificationJob`; group +CSV import suppresses notifications. Follow those current patterns. + +Cohort jobs must use `NotificationDeliveryPolicy.fanout_allowed?` before +processing recipients. Use `NotificationService` rather than creating records +directly, so concurrent event producers share the recipient quota. Inspect the +worst-case audience and every caller of the hook: group membership changes can +also come from tutorial changes, enrolment deletion and CSV import. Never +notify somebody about their own action. + +## Documentation + +All notification documentation lives in `doubtfire-api/docs/notifications/`. +For general documents, use lowercase, hyphenated names with no dates and one +file per subject. Event filenames use the exact lower-snake-case event passed +to `NotificationService.notify`, for example `task_comment_created.md`. + +| What you are writing | Where it goes | +| --- | --- | +| An event | `docs/notifications/events/.md` | +| Anything else | `docs/notifications/.md` | + +Copy [events/_template.md](events/_template.md) and fill in its eight-field +table for every event. Document the trigger, recipient eligibility, preference +category, deduplication identity and safe deep link. The worked example is +[events/task_comment_created.md](events/task_comment_created.md). +Keep final user-facing wording in its mail template or push implementation; +do not maintain a competing copy in documentation. Add recurring contributor +questions and their answers here. + +## Tests and development + +API tests use Minitest under `test/`, mirroring application paths. Use +`test/factories/notification_factory.rb` for events. Web tests use Vitest with +`.spec.ts` next to the component. + +```sh +# In the API development container with a populated, isolated test database: +bundle exec rails test test/sidekiq/notification_email_job_test.rb +# In the web repository: +npx vitest run src/app/api/services/notification.service.spec.ts +``` + +Follow the API contributor guide for Ruby, MariaDB and Redis setup. Test a +normal path, denied/invalid inputs and retries where relevant. Never populate +or clean a shared database while somebody else's tests are using it. + +Development-only `DOUBTFIRE_NOTIFICATION_EMAIL_INLINE=true` permits Mailpit +email testing without a worker. It never enables inline production delivery. +Use synthetic users and redact credentials and student data in PR evidence. +Mailpit capture proves rendering, not inbox delivery. A browser unit test does +not prove that a physical phone displayed a push. + +Read [push-setup.md](push-setup.md) and +[testing-push-locally.md](testing-push-locally.md) before testing on a phone. +A phone reaching `http://192.168.x.x:4200` over Wi-Fi is not in a secure context; +use the HTTPS tunnel described there. On iOS, also Add to Home Screen and open +the app from its icon. diff --git a/docs/notifications/RUNBOOK.md b/docs/notifications/RUNBOOK.md new file mode 100644 index 0000000000..9fff1d86d9 --- /dev/null +++ b/docs/notifications/RUNBOOK.md @@ -0,0 +1,224 @@ +# Notifications production runbook + +Scope: the `11.0.x` API and the production stack in +[`doubtfire-deploy`](https://github.com/ontrack-features-t2-2026/doubtfire-deploy/blob/11.0.x/production/docker-compose.yml). +Commands below run from that repository's `production/` directory and use its +existing private `.env.production`. Confirm the environment before operating. +Do not copy production secrets or student content into incident reports. + +## Stop delivery first + +**There is no global notification kill switch or per-channel runtime toggle in +this revision.** `NotificationService` still creates in-app records. Stopping +Sidekiq contains queued email/push delivery without building a new image: + +```sh +docker compose --env-file .env.production stop -t 60 sidekiq +``` + +This stops **all** work on that service, including `submissions` and `default`. +Check for separately managed workers before relying on it. In-flight messages +may already have reached their provider. New requests can continue queueing +work; monitor Redis capacity. This is a temporary containment action, not a +permanent opt-out or a way to retract email/push already delivered. + +For email-only containment, production supports `DF_MAIL_PERFORM_DELIVERIES=no`. +Set it in the private environment file and recreate both API and worker: + +```sh +docker compose --env-file .env.production up -d --no-deps --force-recreate apiserver sidekiq +``` + +This affects **all Rails mail**, including non-notification mail. Jobs consumed +while delivery is disabled succeed without sending; they are not saved for +later replay. Keep workers stopped instead when delivery must be retained. +A Rails console assignment changes only that console process. A plain Docker +`restart` does not load changed environment values. + +For push-only containment there is no supported production toggle. Removing +VAPID keys makes delivery a no-op in API code, but production validation requires +those keys and doing this would consume queued pushes without sending. Do not +use key rotation as a delivery pause. Keep the worker stopped until an approved +queue configuration or runtime control is available. + +Restore a stopped service with `docker compose --env-file .env.production start +sidekiq` only after reviewing queued work and the original failure. Restoring +email requires setting `DF_MAIL_PERFORM_DELIVERIES=yes` and recreating the +processes again. No source change or image rebuild is needed for these actions. + +## Find the failing stage + +A successful API response is not proof of email or push delivery. + +1. Check API/worker/Redis health with + `docker compose --env-file .env.production ps` and worker logs with + `docker compose --env-file .env.production logs --since 15m sidekiq`. +2. Use the authenticated user's notification list to verify the in-app record. + Category preferences can suppress record creation entirely. +3. Check queues, retries and workers using the read-only console commands below. +4. A `NotificationEmailJob` runs on `mailers`; a + `PushNotificationDeliveryJob` runs on `notifications`. Both retry three times. + The optional additional-address copy has a separate delivery job. +5. After a queue accepts a push job, `Notification.delivered_at` is set. It is a + **push hand-off timestamp**, not an SMTP receipt, push receipt or read receipt. + Email is queued after the notification creation transaction commits. + +Open the console in the API process, which uses the same Redis as the worker: + +```sh +docker compose --env-file .env.production exec apiserver bundle exec rails console +``` + +Then inspect aggregate metadata only: + +```ruby +require 'sidekiq/api' +%w[mailers notifications submissions default].each do |name| + queue = Sidekiq::Queue.new(name) + puts({ queue: name, depth: queue.size, oldest_seconds: queue.latency }.inspect) +end +puts({ retry: Sidekiq::RetrySet.new.size, + scheduled: Sidekiq::ScheduledSet.new.size, + dead: Sidekiq::DeadSet.new.size }.inspect) +Sidekiq::ProcessSet.new.each do |process| + puts({ identity: process.identity, busy: process['busy'], + concurrency: process['concurrency'], queues: process['queues'] }.inspect) +end +``` + +Take two snapshots a minute apart. Falling depth/latency with busy workers +indicates progress. Growing latency with no process consuming the named queue +indicates a missing worker/queue configuration. Busy workers plus provider +errors/retries indicate slow or failing delivery; inspect SMTP/push responses +before increasing concurrency. Size alone cannot distinguish stuck from slow. +Production consumes queues in strict priority order: `mailers`, `notifications`, +`submissions`, `default`; a mail backlog can delay everything below it. + +## Drain, retry or discard + +To drain, stop the event source that caused the burst (for example the import), +fix the provider or worker issue, and let the normal worker process queued work. +Watch depth, latency and retry/dead counts until stable. A continuously active +cohort can prevent an empty queue. Increasing worker concurrency requires +checking database pool, memory and provider limits first. + +To stop taking new work while finishing in-flight jobs, the console can quiet +workers with `Sidekiq::ProcessSet.new.each(&:quiet!)`. This applies to all +Sidekiq processes using that Redis namespace. It does not drain waiting jobs; +restart workers when ready to resume. Do not use `Queue#clear`, `RetrySet#clear` +or Redis `FLUSHDB` as incident recovery. + +Inspect one known job ID without printing its arguments or exception message: + +```ruby +jid = 'replace-with-the-job-id-from-the-incident' +job = Sidekiq::RetrySet.new.find_job(jid) +puts({ jid: job&.jid, klass: job&.klass, queue: job&.queue, + error_class: job&.item&.fetch('error_class', nil), + attempts: job&.item&.fetch('retry_count', nil) }.inspect) +``` + +After the cause is fixed, `job.retry` retries that one job immediately. Delivery +is at-least-once; confirm whether the provider already accepted it, since a +partial failure can duplicate delivery. Do not replay a whole cohort blindly. + +For a confirmed poisoned job, record its JID/class/queue, reason for discard and +whether a replacement notification is required in the incident record. Check +it is not currently executing. Select the exact entry from `RetrySet`, +`DeadSet`, `ScheduledSet` or its named `Queue`, then call `job.delete` only on +that entry. Deletion permanently loses that pending attempt. Do not delete the +`Notification` database record to clear a queue: email jobs skip missing records, +while push jobs raise and retry a missing record. Redis iteration races with workers, so pause the +affected worker before selecting/deleting and confirm the JID is absent after. + +## Mail deliverability and ownership + +Check in this order: + +- Was an in-app record created, and does the recipient still permit that category? +- Is `mailers` being consumed? Did the job enter retry/dead after an SMTP error? +- Is `DF_MAIL_PERFORM_DELIVERIES` exactly `yes`, and is delivery method `smtp`? +- Are SMTP host, port, TLS, authentication and sender domain correct? Compare + deployed configuration privately; never print passwords into a report. +- Does the provider show accepted, deferred, bounced or rejected delivery? + Check the mailbox spam folder after acceptance. An accepted SMTP transaction + does not prove inbox arrival. +- Ask the institution's **mail/DNS administrator** to check SPF, DKIM signing, + DMARC alignment and provider rejection diagnostics for the configured sender. + The application team owns queue/config diagnosis; the institution controls DNS. + +No named institutional DNS owner or escalation address is recorded in these +repositories. Before handover, the deployment owner must record that contact +in the private operations directory, together with the SMTP provider contact +and incident escalation route. This runbook cannot establish ownership from a +repository alone. Do not invent a person or edit DNS during routine diagnosis. +Follow the [email acceptance procedure](email-acceptance.md) for exact +configuration checks, a controlled single-recipient trial and evidence required +to close NPR-Q01. Repository tests do not prove mailbox placement. + +## Push and VAPID rotation + +Follow [push setup: keys](push-setup.md#the-keys) for the existing rotation +procedure and subscription invalidation requirement; do not rotate to solve a +queue backlog. Coordinate the change across API and workers and check the +[production deployment procedure](https://github.com/ontrack-features-t2-2026/doubtfire-deploy/blob/11.0.x/DEPLOYING.md) +for environment validation/recreation. The public key is public; the private +key stays in the secret store. + +For diagnosis, verify keys are configured, the intended user has a browser +subscription, notification permission is granted, and the worker consumes +`notifications`. 404/410 subscription responses remove stale registrations; +other provider errors are retained and raised for retry. Check +[push failure handling](push-setup.md#failure-handling) and +[phone testing](testing-push-locally.md) before assuming the provider is down. + +## Fan-out and deliberate cohort messages + +The default cohort ceiling is 500 candidate recipients; the default per-recipient +quota is 30 external notification hand-offs per rolling 3,600 seconds. The +[delivery operations guide](delivery-operations.md#cohort-and-recipient-limits) +defines the three environment settings, protected event producers, current-row +locking and deliberate operator overrides. A rejected cohort logs +`notifications.fanout_limit`; a recipient over quota retains a `throttled` +in-app record and receives no external channel hand-off. + +Current mitigations are explicit background fan-out jobs, recipient eligibility, +category preferences, duplicate keys and suppression of internal group moves. +See the [recipient amplification review](reviews/recipient_amplification_risk.md) +for the reviewed risks, and current [event documentation](events/README.md) for +recipient rules. That historical review includes planned events; current code +and event docs take precedence. + +For a genuine all-cohort communication, agree the recipient scope and expected +count with the unit owner, review the communication rule, run a small test in +staging, and monitor channel backlogs/provider quotas while executing. Use only +the documented producer-specific override after reviewing the count; keep the +same event/change identity for deduplication. Do not edit individual users' +preferences or disable duplicate protection. The +[cohort benchmark](cohort-load-testing.md) exercises the delivery service +directly and does not authorize a larger production ceiling. + +## Failure modes recorded in this repository + +These are documented defects/setup failures, not a claim that this author +observed production incidents. Dates, impact and incident owners are not +available in the repository. + +| Symptom | Recorded cause and recovery | Evidence | +| --- | --- | --- | +| Queue accepts email/push but nothing arrives | Worker consumed `default` only; configure `mailers` and `notifications` | [`config/sidekiq.yml`](../../config/sidekiq.yml) explains the fixed silent failure | +| Phone cannot enable push over a LAN URL | Browser requires a secure context; use HTTPS and iOS Home Screen flow | [Phone testing](testing-push-locally.md) | +| Development email appears missing | Without SMTP, mail is a file under the deploy mount, not the API checkout | [Development config](../../config/environments/development.rb) | +| Plain-text communication email contains `%>` | Nested ERB comment ended early; restore real greeting/sign-off and regression coverage | [Mailer regression](../../test/mailers/communications_mailer_test.rb), fixed by `42f7373191bb0b6686e0d2d4ee71cc3a2f814f80` | + +## Handover verification still required + +The repository deliverable is this runbook. Production sign-off additionally +requires the missing runtime controls/fan-out limits, a named private escalation +contact, and an independent operator dry-run. No such dry-run is claimed here. + +In staging with mail captured and no real recipients, have someone outside the +notifications team diagnose a paused worker using only this guide. Record the +starting symptom, commands, diagnosis, recovery, operator and confusing steps. +Fold their feedback into a PR. Keep operational evidence private if it includes +addresses, message contents or secrets. diff --git a/docs/notifications/additional-notification-email.md b/docs/notifications/additional-notification-email.md new file mode 100644 index 0000000000..7a50797e44 --- /dev/null +++ b/docs/notifications/additional-notification-email.md @@ -0,0 +1,115 @@ +# Verified additional notification email + +## Scope + +An account may register one optional **Additional notification email**. The +institutional/sign-in address remains the primary notification destination. A verified +additional address receives a separate copy of notification-service email; it is never added as a +CC and it never replaces the primary address. + +This feature extends `NotificationEmailJob`, the existing single-notification path documented in +`docs/notifications/existing-emails.md`. It deliberately does not copy the separate legacy, +operational, weekly-summary, D2L, communication-rule, or portfolio mailers. Adding the address to +those independent paths would create the duplication risks identified by that audit. + +## State and API + +`additional_notification_emails` has one row per user and stores the normalised address, a +monotonic verification version, issue/expiry timestamps, and the verified timestamp. It never +stores a verification token or digest. `additional_notification_email_audits` stores only the user, +an allow-listed event, and timestamps; it stores no address, token, notification body, or mail +content. + +Authenticated, self-only endpoints: + +- `GET /api/users/:user_id/additional_notification_email` +- `PUT /api/users/:user_id/additional_notification_email` +- `POST /api/users/:user_id/additional_notification_email/resend` +- `DELETE /api/users/:user_id/additional_notification_email` + +The public completion endpoint is `POST /api/additional_notification_emails/verify`. The signed +token is submitted in the request body. It is never part of an API URL, and `token` is in the +server parameter-filter list. State responses are marked `private, no-store` and expose only +`none`, `pending`, or `verified`, the requested address, and the expiry time. + +## Verification and abuse controls + +- Links are signed with the Rails secret, purpose scoped, and expire after 24 hours. +- A resend or replacement increments `verification_version`; every earlier link and queued mail + job becomes stale immediately. +- Verification rechecks the signed version while holding the row lock, closing the + replacement/verification race. +- A used link returns a conflict; an expired, replaced, or malformed link fails. +- At most three request/resend operations per account are accepted in a rolling one-hour window. + The limit uses persistent, token-free audit events rather than process memory. +- The address is lower-cased and cannot equal the institutional address case-insensitively. +- Removal destroys the destination, invalidates every outstanding link, and leaves only the + content-free `removed` audit event. +- Background jobs carry stable database ids and the verification version, never an address, token, + subject, or notification content. + +The verification email points to `/verify_additional_email#token=...`. The web bootstrap captures +that fragment into one-use module memory and removes it with `history.replaceState` before Sentry +or Angular starts. The verification page then POSTs it in the request body. This prevents the +secret from entering browser history, referrers, client telemetry, or API access-log URLs. + +## Delivery isolation and preferences + +`NotificationEmailJob` rechecks the current category preference and sends the primary message +first. Only after that message is accepted does it enqueue `AdditionalNotificationEmailDeliveryJob` +with notification id, additional-email id, and verification version. + +The optional job independently rechecks all of the following at execution time: + +1. the destination still belongs to the notification user; +2. it is still verified and has the queued version; +3. it is not equal to the current institutional address; and +4. the current task/feedback/portfolio notification preference still permits delivery. + +It then sends a second message using the event's existing safe template. The optional job has its +own retry lifecycle. Its failure or queue hand-off failure is audited and logged using ids and the +exception class only; it cannot fail or retry the already accepted primary message. This avoids +both primary suppression and primary duplication. + +## Identity boundary + +The profile API treats institution-controlled identity as account data: + +- under SAML/AAF/LDAP-style authentication, a changed email is rejected for self-service and + administrator requests; +- a user cannot change their own student id under any authentication mode; +- under institution-controlled authentication, an administrator cannot forge another user's + student id; and +- database-auth development installations retain local email editing and administrator + maintenance of another local account's student id. + +Sending the unchanged rolling-client values is accepted, so preferred name and genuine profile or +notification settings continue to save. User serialization exposes +`institutional_identity_managed` and `email_editable` so the web does not infer identity policy +from a role or hostname. + +## Deployment order + +Deploy this as one ordered change; do not expose the new UI against an API without its schema. + +1. Back up and migrate the API database with + `20260831012000_create_additional_notification_emails.rb`. +2. Deploy/restart the API so the state, verification, and identity-policy endpoints are present. +3. Deploy/restart a Sidekiq worker using `config/sidekiq.yml`. Both verification and optional-copy + jobs use the existing `mailers` queue, so at least one worker must listen to it. +4. Confirm the configured sender and SMTP provider can deliver the application's domain, then + deploy the web client. +5. Run a pending -> verified -> copied -> removed smoke test. Confirm a category opt-out between + the primary and optional job suppresses the optional copy. + +Rolling back the web and API code is safe while the two new tables remain. Dropping the tables is a +separate destructive database operation and is not part of an application rollback. Rotating the +Rails secret intentionally invalidates every outstanding verification link. + +## Environment verification on 2026-08-31 + +The available API and worker were configured only for the local Mailpit catcher on port 1025, +using a `.local` sender. Mailpit had no outbound relay/provider configuration. One isolated SMTP +handoff to a reserved `example.test` recipient was accepted and stored by Mailpit; all temporary +database writes were rolled back. No external mailbox delivery was attempted or claimed. See the +Batch 12 evidence handover for the message identifiers and exact blocker. diff --git a/docs/notifications/cohort-load-testing.md b/docs/notifications/cohort-load-testing.md new file mode 100644 index 0000000000..2f25585ad7 --- /dev/null +++ b/docs/notifications/cohort-load-testing.md @@ -0,0 +1,200 @@ +# Cohort notification load testing + +`script/benchmark_notifications.rb` provides a reproducible synthetic workload +against a real database and Redis queue. It seeds configurable synthetic users, +valid push subscription material, and a configurable opt-out mix, then compares +inline delivery with queued delivery and two concurrent events. Mail delivery +uses Rails' test transport; push delivery is stubbed at the provider boundary. +It records trigger-service time, queue drain time, total time, sampled queue +depth and Linux resident memory, and removes its synthetic users on completion. +Each mode asserts expected notifications, persisted `delivered` rows, email +attempts, Rails test emails, stubbed push calls and empty queues. Cleanup is +verified before a successful JSON result is emitted. Concurrent peak sampling +uses a mutex. Sampling after notification creation and each drained job adds +overhead that is included in the timings. + +```sh +RAILS_ENV=test COHORT_SIZE=100 OPT_OUT_PERCENT=20 \ + BENCHMARK_SOURCE_REVISION="$(git rev-parse HEAD)" \ + BENCHMARK_RESULT_PATH=/tmp/notification-benchmark.json \ + bundle exec rails runner script/benchmark_notifications.rb +``` + +Use a dedicated populated test database and empty dedicated Redis database +(number 1 or higher). Never run alongside other workers using those queues. +The harness refuses production and nonempty delivery queues. It intentionally +uses only synthetic transports and `.invalid` email addresses. +It sets the recipient quota to 100 in this process; no recipient receives more +than four events across the three modes. It calls `NotificationService` directly, +bypassing cohort producer admission. A 1,000-user result does **not** imply that +the default 500-candidate ceiling permits that cohort in production. + +Set `COHORT_SIZE` to the actual largest enrolment supplied by the institution, +then repeat with headroom and increasing sizes (record that enrolment as +`LARGEST_UNIT_ENROLMENT`). Keep the output and hardware/worker configuration with +the PR. Identify the size where the agreed latency or memory budget fails. +Do not describe a small local run as proof of production capacity. + +The trigger measurement starts at `NotificationService`, so it excludes HTTP, +authentication and event-specific database eligibility queries. Queue drain is +one local worker and provider latency is synthetic. Production acceptance still +requires HTTP timing, the actual largest enrolment, representative worker +concurrency, transport latency, delivery-rate constraints and a measured +saturation point. Those institutional measurements are not fabricated here. + +## Current 500- and 1,000-user results — 21 September 2026 + +Both cohorts completed with 20% opt-outs, all expected synthetic channel deliveries, +empty queues and verified fixture cleanup. Each single event reached 400/800 +eligible users respectively; each two-event case produced 800/1,600 notifications, +emails and pushes. No real message left the test process. These are individual +samples under shared local load, not production capacity or an HTTP SLO result. + +| Users | Mode | Trigger (s) | Drain (s) | Total (s) | Sampled queue peak | Sampled RSS (KiB) | Verified notifications / emails / pushes | +| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | +| 500 | inline_baseline | 15.8606 | 0.0004 | 15.8610 | 2 | 334160 | 400 | +| 500 | queued | 6.6502 | 14.4861 | 21.1363 | 800 | 334624 | 400 | +| 500 | concurrent_queued | 31.9343 | 17.8731 | 49.8074 | 1600 | 349408 | 800 | +| 1000 | inline_baseline | 192.4294 | 0.0018 | 192.4313 | 2 | 338608 | 800 | +| 1000 | queued | 41.6023 | 27.7178 | 69.3201 | 1600 | 339112 | 800 | +| 1000 | concurrent_queued | 52.5015 | 51.7298 | 104.2313 | 3200 | 361544 | 1600 | + +[Raw 500-user results](benchmark-results/2026-09-21/cohort-500.json), +[raw 1,000-user results](benchmark-results/2026-09-21/cohort-1000.json) and +[exact revisions, isolation, resource limits and reproduction commands](benchmark-results/2026-09-21/README.md) +are committed together. Runtime code is unchanged from `0fee2ba`; the harness +adds verified counts and synchronized sampling. The actual institutional maximum, +a real pre-queue comparison, HTTP latency and a saturation point remain unmeasured. + +## Find the real cohort size without student data + +An authorized deployment owner runs this read-only query in the deployed API. +It follows `Unit#active_projects` (`enrolled: true`) and the active unit scope +used by scheduled notification producers. Only counts are printed. + +```sh +bundle exec rails runner - <<'RUBY' +counts = Project.joins(:unit).where(enrolled: true, units: { active: true }) + .group(:unit_id).count.values +puts({ active_units_with_enrolments: counts.length, + largest_active_unit_enrolment: counts.max || 0 }.to_json) +RUBY +``` + +The owner confirms that active units cover the intended teaching period. The +registrar/unit owner supplies upcoming enrolments and forecast growth; a current +database maximum is not a forecast. Record the approved maximum in +`LARGEST_UNIT_ENROLMENT` when rerunning. A contributor need not know these values +or have production credentials. Obtain the staging URL and release configuration +from the hosting/deployment inventory, not guessed repository URLs. + +## Finish institutional capacity acceptance (NPR-T01) + +The deployment and service owners fill in these thresholds before testing. +An unknown threshold is an open acceptance input, not a pass. + +| Input | Owner decision | +| --- | --- | +| Cohort and growth | Current/forecast maximum, teaching period, headroom | +| Event mix | Release, due-date change and reminders; burst/overlap frequency | +| HTTP budget | p95/p99 trigger latency, error rate and measurement window | +| Delivery budget | Maximum queue age/drain time, sustainable notifications/second | +| Resources | API/worker/DB CPU and memory, DB pool/connections, reserve | +| Transports | Worker count/concurrency, provider rate limits and sandbox latency | +| Saturation plan | Warm-up/repeat count, size/concurrency increments, stop conditions | + +1. Create an isolated staging fixture with synthetic enrolled users, an active + unit and task definitions at the approved maximum/opt-out mix. Include target + grades, withdrawn enrolments and dates. Use approved provider sandboxes/sinks + and controlled browser subscriptions. Verify every worker's transport boundary: + the harness's in-process push stub does not affect a separate Sidekiq process. + Never fan out to real students for a load test. +2. Measure the real authenticated convenor trigger routes: + `POST /api/units/:unit_id/task_definitions/` creates a task release; + `PUT /api/units/:unit_id/task_definitions/:id` with changed `task_def[due_date]` + creates a due-date event. Capture HTTP duration/status in the load tool or + reverse-proxy metrics. Fresh events need fresh fixture definitions/change + identities; replaying an identity intentionally deduplicates and produces + misleadingly cheap measurements. Preserve identities when testing retries. +3. Measure the event-specific eligibility/database work and worker queues. The + routes enqueue `NewTaskAvailableNotificationJob` and + `TaskDueDateChangedNotificationJob`. In the isolated fixture environment, + scheduled events can be queued using + `SendNewTaskAvailableNotificationsJob.perform_async` and + `SendDueSoonRemindersJob.perform_async` in Rails console. These sweep every + active unit, so never share a database with real students. Arrange fixture + dates/markers to satisfy each job's eligibility/catch-up window, and verify + expected eligible and opted-out counts before accepting timing. +4. Verify admission/rejection first. Above the configured ceiling, expect + `notifications.fanout_limit` and no cohort delivery. A deliberate larger trial + uses the producer-specific override in + [delivery operations](delivery-operations.md#cohort-and-recipient-limits), + records it and preserves the original event identity for retries. This does + not demonstrate that normal requests bypass the limit. +5. Start the release's actual worker command. Record effective queues, + `DF_SIDEKIQ_CONCURRENCY`, DB pool sizes and container limits. Capture queue + depth/age, retry/dead counts, CPU/RSS and DB utilization at fixed intervals. + Use `docker stats` or the deployment monitor for all services; the harness's + RSS is one Ruby process, not the total memory footprint. +6. Repeat the same workload, then increase size/event overlap/worker concurrency + using the agreed matrix. Include two distinct simultaneous events, peak burst + and a controlled transient provider failure/recovery. Stop at the first + agreed latency/error/memory/provider boundary and let queues drain. Record + the last passing and first failing point; a run that never reaches a boundary + cannot claim saturation. +7. Compare against a real pre-queue checkout under comparable fixture, hardware, + cache and transport conditions. The parent of the email queue introduction + (`67d9f92e`) is `564ff793f899e10fbfb4eaa80cfa3616da8e62ed`; source inspection + confirms `NotificationService.notify` delivered both channels inline there. + Create a separate detached worktree with + `git worktree add --detach ../notification-prequeue-baseline 564ff793f899e10fbfb4eaa80cfa3616da8e62ed`, + and use its own disposable database, locked dependencies and synthetic capture + adapters. Adapt the measurement driver to that revision's schema/API rather + than applying current migrations to it. The current harness's inline mode is + a synchronous comparison on current code, **not** this historical measurement. + Record p95/p99 over the agreed repeats, counts, drain/queue age, retries/failures, + opt-outs, exact commands/revisions and acceptance decision. + +Read-only queue snapshot in the selected staging API environment: + +```sh +bundle exec rails runner - <<'RUBY' +require 'sidekiq/api' +queues = %w[mailers notifications default].to_h do |name| + queue = Sidekiq::Queue.new(name) + [name, { depth: queue.size, oldest_wait_seconds: queue.latency }] +end +puts({ queues: queues, retry_count: Sidekiq::RetrySet.new.size, + dead_count: Sidekiq::DeadSet.new.size }.to_json) +RUBY +``` + +Institutional maximum, HTTP SLOs, real provider latency and saturation remain +unverified until these operator steps are executed and reviewed. + +## Earlier harness smoke — 21 September 2026 + +A single smoke run completed with 40 synthetic recipients, 20% opted out, a +dedicated MariaDB 12.3 test database and Redis database 9. The API image was +`ontrack-unit-hub-release-preview-api:20260914` (Ruby 3.4.10, Rails 8.0.5.1). +These measurements used API source `e95bd592`, before the later rebase and +recipient-update compatibility fix; they are harness evidence, not a timing +claim for every subsequent API revision. +The Docker VM exposed 10 CPUs and 7.75 GiB memory; the API container had no +additional CPU or memory limit. Other local test containers shared that VM. + +| Mode | Simultaneous events | Trigger service (s) | Queue drain (s) | Total (s) | Sampled queue peak | Sampled RSS (KiB) | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Inline baseline | 1 | 2.2752 | 0.0003 | 2.2755 | 2 | 317024 | +| Queued | 1 | 0.1893 | 0.3195 | 0.5088 | 64 | 317096 | +| Concurrent queued | 2 | 0.2697 | 0.5048 | 0.7745 | 128 | 322320 | + +Both delivery queues were empty afterward and all synthetic users were removed. +This verifies that the harness runs through opt-outs, two external channels and +concurrent events. These are single samples in the displayed order without a +warm-up, so cold-start and cache effects differ between modes. The inline mode +drains after each recipient through the same queue adapters; it is a synchronous +comparison within this harness, not a measurement of a historical application +version. No email or push message left the test process. The institution's +largest enrolment was unavailable, and this run establishes neither a supported +cohort limit nor production latency, throughput or saturation. diff --git a/docs/notifications/delivery-operations.md b/docs/notifications/delivery-operations.md new file mode 100644 index 0000000000..50e64962ce --- /dev/null +++ b/docs/notifications/delivery-operations.md @@ -0,0 +1,119 @@ +# Notification delivery operations + +## Email outcomes and retry + +`Notification.email_delivery_state` records `pending`, `delivered`, `retrying`, +`failed`, `queue_failed`, `suppressed` or `throttled`. Historical rows are +`untracked`: their past SMTP outcome cannot be reconstructed. Attempt count, +SMTP acceptance time and exception class are persisted without addresses, +message content or SMTP response text. SMTP acceptance does not prove inbox +placement or capture a later bounce. + +Primary mail uses the existing `mailers` Sidekiq queue with three retries +(four total attempts). Transient failures use Sidekiq's exponential backoff +with jitter. SMTP fatal/syntax errors (5xx permanent failures) go immediately +to the dead set. Exhausted retries persist `failed`, even after the dead set is +cleared. Deleted notifications are skipped; the after-commit enqueue prevents +a worker observing a not-yet-committed row. Opt-out and the configured mail-delivery switch are checked again at delivery; +disabled mail is recorded as `suppressed`, never `delivered`. Notification mailers +raise transport errors even when development mail normally swallows them. + +```sh +bundle exec rake notifications:delivery_counts +NOTIFICATION_ID=123 bundle exec rake notifications:retry_email +``` + +Investigate and correct the recipient/configuration first. The retry task only +accepts `failed` or `queue_failed` records. Successful delivery is guarded by a +row lock and persisted completion marker. SMTP cannot provide exactly-once +delivery: a process crash after SMTP acceptance and before the database update +can still duplicate a message. Additional verified-address copies retain their +independent audit/retry lifecycle so a copy failure never repeats primary mail. + +## Cohort and recipient limits + +| Variable | Default | Meaning | +| --- | --- | --- | +| `DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT` | 500 | Maximum candidate recipients without explicit operator override | +| `DOUBTFIRE_NOTIFICATION_RECIPIENT_LIMIT` | 30 | Maximum external notification hand-offs per recipient per window | +| `DOUBTFIRE_NOTIFICATION_RECIPIENT_WINDOW_SECONDS` | 3600 | Rolling quota window | + +All values must be positive integers. Fanout counts are conservative candidate +project counts before detailed eligibility/preferences; they may overcount but +cannot undercount the candidate cohort. `notifications.fanout_limit` records +event, trigger id, candidate count, limit and admission decision. No names, +addresses or message bodies are logged. The recipient row lock serializes +quota reservations across concurrent event producers. Recipient, quota and +deduplication lookups use current locking reads even within an older caller +transaction. MariaDB versions that expose `innodb_snapshot_isolation` can reject +both those reads and the notification insert's foreign-key locks when a +recipient was updated after the caller's snapshot. During reservation only, +the connection temporarily disables that snapshot-conflict check and restores +its previous value before the surrounding workflow resumes, including on +exceptions. Row locks, foreign-key constraints and outer rollback remain in +force; ordinary MySQL is unchanged. Outside that reservation block, explicit +current reads use a statement-only override. An existing event's pending push +handoff is retried after the caller commits, preventing a retry from updating +an unseen row. In-app records remain available for throttled events, with no +push/email queued. + +The compatibility setting follows MariaDB's documented dynamic session scope: +[InnoDB snapshot isolation](https://mariadb.com/docs/server/server-usage/storage-engines/innodb/innodb-system-variables#innodb_snapshot_isolation). +It is not a global database setting or a change to the deployment's default +transaction isolation. + +An operator can deliberately rerun a verified cohort from Rails console, for +example `NewTaskAvailableNotificationJob.perform_async(task_definition_id, true)`. +The due-date job accepts its override as the fifth argument after change id; +retain the original change id to preserve deduplication. Scheduled due-soon and +availability sweeps accept a single boolean argument. Never enable an override +from an untrusted request parameter. Inspect recipient counts before use. + +## VAPID secrets and rotation + +Production API and worker boot require a matching P-256 VAPID public/private +pair. Development with both unset disables push. Invalid/partial keys fail +with a diagnostic naming the configuration, never the key material. Only the +public key appears on the authenticated settings response. Inject the private +key at runtime from the deployment's secret manager; never put it in Git, +Docker build arguments, image layers, support tickets or logs. + +Rotation changes the application-server key. Existing subscriptions remain +bound to the old key and cannot be used with the replacement pair. Treat a +leaked private key as compromised immediately; do not keep it for convenience. + +1. Generate a fresh pair in the controlled deployment environment using + `WebPush.generate_key`; store it directly in the secret manager, without + printing the private value to CI output or saving it in the repository. +2. Stage both values together and verify startup in a private environment. +3. Pause the push worker, deploy the pair to API and every worker, and restart + them consistently. A mixed pair or mixed rollout produces failed deliveries. +4. In a controlled Rails console, invalidate old rows with + `PushSubscription.delete_all`. This deletes subscriptions, not notifications. +5. Resume the worker and instruct users to open notification settings and + re-enable push. The MN-C05 client resubscription flow is required; an old + browser registration may need to be unsubscribed before it can be recreated. +6. Verify an authenticated subscription and a synthetic notification on each + supported browser. Record only key version identifiers, never private values. + +Repository/image/log scanning must be performed against the actual deployed +secret and built release by its owner; source review cannot attest to all +historic image layers or external logs. The repository's Docker ignore rules +exclude local `.env` files and key files. No real production secret was used +for this change's synthetic validation. + +## Structured operation logs + +`projects.index` emits once per authenticated collection request: `user_id`, +`include_task_definitions`, `include_inactive`, `project_count` and +`task_definition_count`. The latter counts definitions already preloaded for +returned projects, so it does not issue a second database query. Summary-only +responses report zero definitions. Use it alongside normal request duration +to compare dashboard and summary traffic; it does not log task contents. + +`units.bulk_withdraw` records actor `user_id`, `unit_id`, row/result counts and +successfully withdrawn `project_ids`. `units.csv_export` records actor and unit. +Ids avoid duplicating student names/emails into logs. The JSON id list grows +linearly (300 numeric ids are typically a few kilobytes); configure log storage +to preserve complete lines, because truncation loses recovery evidence. Denied +requests do not produce a success audit event. diff --git a/docs/notifications/email-acceptance.md b/docs/notifications/email-acceptance.md new file mode 100644 index 0000000000..4a40d2d8ea --- /dev/null +++ b/docs/notifications/email-acceptance.md @@ -0,0 +1,197 @@ +# Email delivery acceptance (NPR-Q01) + +The repository verifies queuing, preferences, retries and persisted SMTP outcomes. +Closing production deliverability acceptance additionally requires an authorized +operator, the deployed mail provider and an institution-controlled test inbox. +Prior Mailpit capture proves rendering and local SMTP hand-off, not delivery to +a real provider. The current benchmark uses Rails' test transport, which also +cannot establish deliverability. No real email was sent for this procedure. + +## Find the deployment and owners + +The deployment owner obtains the target URL, image/source revisions and runtime +configuration from the institution's hosting inventory and deployment records. +Use a private staging installation with production configuration first. Record +the application/deployment owner, SMTP provider administrator and mail/DNS owner +in the private operations directory. The repository does not identify those +people, a staging URL or an approved inbox; a student contributor need not guess +them. The provider administrator can identify the authenticated SMTP service, +verified sender/domain, DKIM selectors, rate limits and delivery-report access. + +Have the institution provision ordinary test users with controlled **primary** +inboxes at Outlook, Gmail and the university's own mail service. Record their +numeric user IDs privately. Keep feedback enabled and give each no verified +additional email, so each trial has one destination. Do not repoint an actual +student's email. Repeat the single-recipient procedure below for all three. + +Run commands below inside the chosen API release container, with its existing +runtime secrets injected. Do not paste credentials or complete environment dumps +into shell history, issues or PRs. These are operator commands, not commands run +as part of writing this document. + +## Check configuration without sending + +```sh +bundle exec rails runner - <<'RUBY' +settings = ActionMailer::Base.smtp_settings +puts({ + rails_environment: Rails.env, + perform_deliveries: NotificationsMailer.perform_deliveries, + delivery_method: NotificationsMailer.delivery_method, + smtp_address: settings[:address], smtp_port: settings[:port], + smtp_domain: settings[:domain], authentication: settings[:authentication], + starttls_auto: settings[:enable_starttls_auto], + username_configured: settings[:user_name].present?, + password_configured: settings[:password].present?, + sender: Doubtfire::Application.config.institution[:email_sender] +}.to_json) +RUBY +bundle exec rake notifications:delivery_counts +``` + +Production SMTP should have deliveries enabled, method `smtp`, the provider's +approved host/port/TLS/auth settings and a verified institutional sender. +Relevant variables are `DF_MAIL_PERFORM_DELIVERIES=yes`, +`DF_MAIL_DELIVERY_METHOD=smtp`, `DF_SMTP_ADDRESS`, `DF_SMTP_PORT`, `DF_SMTP_DOMAIN`, +`DF_SMTP_USERNAME`, `DF_SMTP_PASSWORD`, `DF_SMTP_AUTHENTICATION`, +`DF_SMTP_ENABLE_STARTTLS_AUTO` and `DF_INSTITUTION_EMAIL_SENDER`. Compare the safe +output privately with the provider settings; never print the SMTP settings hash. + +Use `RAILS_ENV=production` in the isolated acceptance installation when validating +production sender headers. Although Rails' `staging` environment loads production +SMTP configuration, `ApplicationMailer` chooses the institutional From header +only when `Rails.env.production?` is true. Do not change the environment of an +unrelated running installation just to perform this test. + +Confirm the deployed Sidekiq process consumes `mailers`; inspect its startup +command/configuration and the [queue checks](RUNBOOK.md). The normal production +worker reads `config/sidekiq.yml`; command-line `-q` flags can override that list. + +## Send one approved email, then verify every boundary + +Only the authorized operator executes this step after verifying the intended +environment and inbox. Set `TEST_USER_ID` to that account and `ACCEPTANCE_TAG` to +a unique, non-personal trial identifier in the operator session. Retain the tag +if retrying the command: the dedupe key prevents creating another event. + +```sh +bundle exec rails runner - <<'RUBY' +abort 'Use the designated production-mode acceptance installation' unless Rails.env.production? +abort 'SMTP deliveries must be enabled' unless NotificationsMailer.perform_deliveries && NotificationsMailer.delivery_method == :smtp +user = User.find(Integer(ENV.fetch('TEST_USER_ID'), 10)) +abort 'Test user must permit feedback notifications' unless user.receive_feedback_notifications +abort 'Use an account without a verified additional email' if user.additional_notification_email&.verified? +tag = ENV.fetch('ACCEPTANCE_TAG') +abort 'Use a short, non-personal trial tag' unless tag.match?(/\A[a-zA-Z0-9_-]{1,80}\z/) +notification = NotificationService.reserve( + user: user, type: 'feedback', event: 'operator_email_acceptance', + message: 'Synthetic email delivery verification', link: '/notifications', + dedupe_key: "operator_email_acceptance:#{tag}:#{user.id}" +) +abort 'Notification was suppressed by the current preference' unless notification +puts({ notification_id: notification.id, email_delivery_state: notification.reload.email_delivery_state }.to_json) +RUBY +``` + +`reserve` creates the in-app row and queues email after commit. This procedure +does not call `deliver`, so it does not queue push. If the result is `throttled`, +wait for the account's quota window or choose a fresh approved test account; +do not bypass the recipient quota. Record the resulting notification ID. + +Set `NOTIFICATION_ID` to that ID and inspect only its non-content audit fields: + +```sh +bundle exec rails runner - <<'RUBY' +n = Notification.find(Integer(ENV.fetch('NOTIFICATION_ID'), 10)) +puts n.attributes.slice('id', 'email_delivery_state', 'email_delivery_attempts', + 'email_delivered_at', 'email_delivery_error_class').to_json +RUBY +``` + +In the controlled user's browser, confirm the same event appears at +`GET /api/notifications` and contributes to `GET /api/notifications/unread_count`. +The normal signed-in client supplies `Username` and `Auth-Token` headers; do not +put tokens in URLs or screenshots. There is no public notification-create API. + +Then inspect the provider's delivery report and the actual inbox/spam folder. +Record the UTC timestamps, provider message identifier, SMTP acceptance, provider +delivery/bounce status and mailbox placement. `email_delivery_state=delivered` +means the SMTP call succeeded; it does not mean inbox arrival. This application +does not persist a provider message identifier or consume later bounce reports. +The operator correlates the trial's time, controlled recipient and message in +the provider console, storing raw headers privately. For Azure Communication +Services, use its documented [delivery reports](https://learn.microsoft.com/en-us/azure/event-grid/communication-services-email-events). + +For an actual failure, diagnose the cause using [delivery operations](delivery-operations.md#email-outcomes-and-retry). +After the cause is corrected, the existing command below requeues only a +`failed` or `queue_failed` notification and **will send email** through the worker: + +```sh +bundle exec rake notifications:retry_email +``` + +It uses the `NOTIFICATION_ID` already set above. Do not replay a delivered row or +an entire cohort. Use a provider-approved controlled recipient/sandbox for the +rejection/bounce trial; do not send to a guessed address. Record who receives +the bounce, the provider status and the current operational response. If no +bounce-handling workflow exists, state that explicitly and raise a separate +follow-up issue with an owner and acceptance criteria; do not implement a new +bounce system as part of this verification ticket. A `failed` SMTP audit state +alone is not evidence of handling a later asynchronous bounce. + +## Check domain authentication and record acceptance + +The provider/mail owner supplies the envelope sender domain, DKIM signing domain +and selector, and visible From domain. These can differ; do not infer them from +`DF_SMTP_DOMAIN` (the SMTP HELO domain). Query their published records read-only: + +```sh +dig +short TXT "$ENVELOPE_SENDER_DOMAIN" +dig +short CNAME "$DKIM_SELECTOR._domainkey.$DKIM_SIGNING_DOMAIN" +dig +short TXT "$DKIM_SELECTOR._domainkey.$DKIM_SIGNING_DOMAIN" +dig +short TXT "_dmarc.$FROM_DOMAIN" +``` + +The mail/DNS owner checks the received message's `Authentication-Results` for +SPF, DKIM and DMARC, including alignment to the visible From domain. A DNS record +existing is insufficient evidence of successful authentication or inbox delivery. +Use the provider's [sender authentication guidance](https://learn.microsoft.com/en-us/azure/communication-services/concepts/email/email-domain-and-sender-authentication) +when Azure Communication Services is the configured provider. DNS changes belong +to the institution's DNS owner and its change process. + +Inspect the received From and Reply-To headers on every trial. The visible +identity must be recognizable to students. Record whether Reply-To (or From +when Reply-To is absent) is monitored; an unmonitored destination must be clearly +identified to recipients. Record the mail/service owner's decision rather than +assuming a configured sender is recognizable or monitored. + +Required provider matrix; fill every row from a real controlled trial: + +| Provider | UTC trial / notification ID | Provider status | Landing zone: inbox / promotions / spam / absent | SPF / DKIM / DMARC and alignment | From / Reply-To identity and monitoring | +| --- | --- | --- | --- | --- | --- | +| Outlook | Pending operator trial | Pending | Pending | Pending | Pending | +| Gmail | Pending operator trial | Pending | Pending | Pending | Pending | +| University mail | Pending operator trial | Pending | Pending | Pending | Pending | + +The ticket's acceptance target is inbox placement across all three providers +with authentication checks passing. Record promotions/spam honestly as a +different landing zone and resolve or obtain the institutional acceptance +decision; do not label a provider's SMTP acceptance as inbox placement. + +Keep this evidence table in the private acceptance record; publish only a +redacted result and its approved evidence reference with the ticket: + +| Required evidence | Operator records | +| --- | --- | +| Tested deployment | Environment/URL, API/web/deploy revisions, UTC date | +| Ownership | Application, SMTP provider and mail/DNS contacts | +| Controlled trial | Trial tag, user/notification IDs, approved inbox reference | +| Application and provider | State/attempts, provider message ID/status, inbox or spam result | +| Authentication | Verified sender, SPF/DKIM/DMARC results and alignment | +| Failure handling | Controlled rejection/bounce result, or explicit unexercised case and owner | +| Acceptance | Defects/actions, evidence location, operator and institutional reviewer sign-off | + +NPR-Q01 remains awaiting institutional acceptance until all three provider rows, +sender identity/monitoring and bounce handling (or its separate follow-up issue) +are evidenced. +A green unit test, synthetic transport or SMTP acceptance alone cannot close it. diff --git a/docs/notifications/events/README.md b/docs/notifications/events/README.md new file mode 100644 index 0000000000..adf511dc53 --- /dev/null +++ b/docs/notifications/events/README.md @@ -0,0 +1,52 @@ +# Notification events + +Every notification event gets its own file in this folder. One event, one file, +named after the event. + + docs/notifications/events/task_comment_created.md + docs/notifications/events/task_status_changed.md + docs/notifications/events/extension_granted.md + +The file name is the string passed as `event:` to `NotificationService.notify`. +If the code says `event: 'task_comment_created'` then the file is +`task_comment_created.md`. No other naming, no grouping by category, no folders +inside this one. + +## Why one file each + +Eight event tickets run at the same time. If they all documented their event in +one shared file, every one of them would edit the same lines and every one would +conflict with the other seven. The first to merge wins and the other seven stop +to fix a merge by hand, for a docs change that had nothing to do with anyone +else's work. + +Adding a file conflicts with nothing. Two people can add +`task_status_changed.md` and `extension_granted.md` on the same afternoon and +neither branch touches the other. + +This is the same reason the mailer looks its template up by event name instead +of holding a lookup table. A new event adds +`app/views/notifications_mailer/.html.erb` and `.text.erb` and edits no +existing file. The docs follow the code. + +## Adding one + +1. Copy `_template.md` to `.md`. +2. Fill in the eight fields. Read the code and copy the real values out of it, + do not write down what you think it does. +3. Delete the field guidance and the worked example from your copy. Both are + there to be read once. Carrying them into every event file is the duplication + this folder exists to avoid. + +The underscore on `_template.md` keeps it at the top of the listing and marks it +as not being an event. The worked example inside it is an example and not the +record for that event, so one event one file still holds. Nothing reads this +folder in code, so the underscore is only for people. + +## What this folder is not + +It is not the design of the notification system. That is `NOTIFICATIONS.md` at +the repo root, and it covers the service, the types, the preferences and the +channels. A file in here is the record of one event: what sets it off, who hears +about it, and where to find the line that raises it. Keep the general +explanation out of it, there is one copy of that already. diff --git a/docs/notifications/events/_template.md b/docs/notifications/events/_template.md new file mode 100644 index 0000000000..5b5ff2a986 --- /dev/null +++ b/docs/notifications/events/_template.md @@ -0,0 +1,118 @@ +# Event: + +| Field | Value | +|---|---| +| Event name | | +| Category | | +| What triggers it | | +| Who receives it | | +| Preference that gates it | | +| Email subject | Required — add the event-specific subject text | +| Email body summary | | +| Where it is raised | | + +## What goes in each field + +**Event name.** The exact string passed as `event:` to +`NotificationService.notify`. Lower case with underscores. It is also the file +name of this document and the name of the mailer templates, so get it right +once and it lines up everywhere. + +**Category.** The `type:` argument. One of `task`, `feedback`, `portfolio`, +`extension`, `general`, from `Notification::TYPES` in +`app/models/notification.rb`. The category is what the user's preference +switches on, so pick the one that matches how a user would think about turning +this off. + +**What triggers it.** The thing a person did, in a sentence. Then the method +that runs afterwards. "A tutor saves a text comment" is more use to the next +reader than "the comment callback fires". + +**Who receives it.** The `user:` argument, and how it is worked out. Say plainly +if it can be nil and what happens then. Most of the bugs in this area are a +recipient that was assumed to exist. + +**Preference that gates it.** The user column in +`Notification::PREFERENCE_FOR_TYPE` that the category maps to, spelled out in +full. Write `none, always sent` for `extension` and `general`, which have no +entry there. When the preference is off the notification is dropped on every +channel, the in-app bell included. + +**Email subject.** What lands in the inbox. Every wired event must have a +human-readable entry in `NotificationsMailer::SUBJECTS`. Record the event-specific +text here, without hard-coding the product name. Unknown or unwired events fall +back to `New notification`. + +**Email body summary.** Two or three lines on what the email tells the reader, +and what it leaves out on purpose. Name the templates. If the event has no +templates of its own say so, the mailer falls back to the generic +`single_notification` pair and the email is much plainer. + +**Where it is raised.** `path/to/file.rb:`, the method it sits in, and +what calls that method. Line numbers move, so name the method too, that is the +part a reader can still find in six months. + +Anything else worth knowing goes below the table under its own headings. +Recipient guards, things left out on purpose, how to check it by hand, the test +file. Keep it short. + +--- + +# Worked example + +This is `task_comment_created`, the first event wired into OnTrack, from ticket +EN-E01. The code it describes lives on `email/task-comment` until that branch +merges, so read the paths below there. Copying the template gives you a copy of +this section and of the guidance above it. Delete both from your own file. + +| Field | Value | +|---|---| +| Event name | `task_comment_created` | +| Category | `feedback` | +| What triggers it | Someone saves a text comment on a task. `Task#add_text_comment` saves the comment and then calls `notify_comment_recipient` | +| Who receives it | `comment.recipient`, set by `add_text_comment` at `task.rb:945` and read here rather than worked out again. The student when a tutor commented. When a student commented it is `Project#tutor_for`, which gives the tutorial's tutor, or the unit's main convenor when there is no tutorial or the tutorial has no tutor | +| Preference that gates it | `receive_feedback_notifications` | +| Email subject | `#{product name}: New task comment`, using the `task_comment_created` entry in `NotificationsMailer::SUBJECTS`. The product name is configured separately, so the event-specific part is `New task comment`. | +| Email body summary | Greets the user by name, gives one line saying who commented on which task in which unit, then says the comment is not included and to open the task to read it. A link to the task and a line about turning the emails off. Templates are `app/views/notifications_mailer/task_comment_created.text.erb` and `.html.erb` | +| Where it is raised | `app/models/task.rb:971`, in `Task#notify_comment_recipient`, called from `add_text_comment` at line 949 | + +## Notes + +The comment text never goes into the message or the email. The email is a prompt +to come back to OnTrack, not a copy of the conversation. The message is built as +`"#{comment.user.name} commented on #{task_definition.abbreviation} in #{unit.code}."` +and the link is `/projects//dashboard/`. + +Raising a notification must not stop a comment being posted, so +`notify_comment_recipient` rescues `StandardError`, logs it and carries on. That +is a second layer. `NotificationService.deliver_email` already rescues mail +failures on its own. + +`notify_comment_recipient` returns early on a blank recipient. The comment in +the code says that happens when the project has no tutor, which is not right, +`Project#tutor_for` falls back to the main convenor. The guard is cheap +insurance rather than a case anyone has hit, and the test for it passes a nil +recipient in by hand instead of going through `add_text_comment`. + +The subject is event-specific. Each wired event must add a short human-readable +entry to `NotificationsMailer::SUBJECTS`; unknown events keep the generic +`New notification` fallback so they can still send safely. + +## How to check it by hand + +1. Sign in as a tutor, open a student's task and post a comment. +2. Development mail is written to a file, and `development/docker-compose.yml` + mounts `../data/tmp` over `/doubtfire/tmp`, so it lands on the host under + `doubtfire-deploy/data/tmp/mails/`. The file is named after the recipient's + address and every email to that address is appended to the same one. Open the + student's file and read the last message. It names the commenter and the task + and does not contain the comment text. +3. Turn that student's feedback notifications off in their profile and comment + again. Nothing is appended. Do not go looking for a new file, there is only + ever the one per address. + +## Tests + +`test/models/notification_task_comment_test.rb`. Both directions, the preference +switch, the missing recipient, the comment text staying out of the email, and a +notification failure still leaving the comment saved. diff --git a/docs/notifications/events/discussion_request_created.md b/docs/notifications/events/discussion_request_created.md new file mode 100644 index 0000000000..c991bb1585 --- /dev/null +++ b/docs/notifications/events/discussion_request_created.md @@ -0,0 +1,85 @@ +# Event: discussion_request_created + +| Field | Value | +|---|---| +| Event name | `discussion_request_created` | +| Category | `feedback` | +| What triggers it | A tutor raises an audio discussion request for a student's task. `Task#add_discussion_comment` saves the request and every audio attachment before notifying. | +| Who receives it | The student whose task owns the discussion (`discussion.recipient`, set to `project.student`). | +| Preference that gates it | `receive_feedback_notifications` | +| Email subject | `#{product name}: New notification`, using the existing `NotificationsMailer#single_notification` subject | +| Email body summary | Tells the student that a discussion prompt is ready and links back to the task. Audio and prompt content are omitted. Templates are `app/views/notifications_mailer/discussion_request_created.text.erb` and `.html.erb`. | +| Where it is raised | `app/models/task.rb`, in `Task#add_discussion_comment`, after the discussion record and all prompt attachments have been saved | + +## Product decision requested for EN-V08 + +EN-V08 was named "Email when a discussion or check-in is booked". OnTrack has +no booking, appointment, or calendar record for a discussion or check-in, so +there is no truthful booking event to raise. + +This implementation proposes the closest existing action with the same user +intent: a tutor creating an audio discussion request through +`Task#add_discussion_comment`. The event is named +`discussion_request_created` to distinguish it from the existing +`DiscussionPrompt` model and its task-definition prompt-management API. The +email says that a prompt is ready and never claims that a meeting was booked. + +The Email Notifications lead should confirm this replacement event before the +PR merges. If the replacement is rejected, close the PR without merging it. + +`Task#add_discussed_comment` and `Task#add_checked_in_comment` are not hook +points. They record that an interaction already happened, in the past tense. + +## Notification fields + +- Type: `feedback` +- Event: `discussion_request_created` +- Recipient: `project.student` +- Message: `A discussion prompt is ready for you.` +- Link: `/projects/:project_id/dashboard/:task_abbreviation` +- Preference: `receive_feedback_notifications` + +The message deliberately leaves out the tutor's name, task name, unit, audio, +and prompt content. The task route is used so the student can act without +searching for the prompt. + +## Delivery timing and failure handling + +The notification is raised only after every uploaded prompt has been accepted, +converted, and attached. A rejected or failed attachment therefore does not +produce a premature email. A request with multiple audio prompts still produces +one notification after the final attachment succeeds. + +`Task#notify_discussion_request_recipient` rescues notification errors so an +email or notification failure cannot undo an already-created discussion +request. `NotificationService` separately handles email-channel failures. + +## Preference behaviour + +The proposed replacement is categorised as `feedback`, matching the existing +tutor-to-student `task_comment_created` event. A student who turns off feedback +notifications receives no in-app, email, or push notification for this event. + +The original EN-V08 ticket suggested `general`, which would always deliver and +could not be opted out of. That always-on behaviour also required a lead +decision, so the proposal uses the safer existing preference until product +owners decide otherwise. + +## How to check it by hand + +1. As a tutor, open a student's task and create an audio discussion prompt. +2. Confirm the student receives exactly one notification and one email after + every prompt upload completes. +3. Confirm the email links to the task but does not contain the audio, prompt + content, task or unit name, or tutor name. +4. Repeat with feedback notifications turned off and confirm nothing is sent. +5. Confirm marking a task as discussed or checked in does not send this event. + +## Tests + +`test/models/notification_discussion_request_test.rb` + +The tests cover a real valid audio attachment, one notification for multiple +audio prompts, no notification after a failed attachment, recipient, event and +category, feedback preference gating, push link, event-specific email copy, +assessment-content omissions, and failure isolation. diff --git a/docs/notifications/events/extension_assessed.md b/docs/notifications/events/extension_assessed.md new file mode 100644 index 0000000000..bb757e955a --- /dev/null +++ b/docs/notifications/events/extension_assessed.md @@ -0,0 +1,35 @@ +# Event: extension_assessed + +| Field | Value | +| --- | --- | +| Event name | `extension_assessed` | +| Category | `extension` | +| What triggers it | A tutor or the automatic extension flow assesses an extension request through `ExtensionComment#assess_extension`. | +| Who receives it | `project.student`. The notification is only raised after the assessment is successfully saved. | +| Preference that gates it | `none, always sent` | +| Email subject | `#{product name}: New notification`, built by `NotificationsMailer#single_notification`. | +| Email body summary | Tells the student whether the extension was granted or rejected. A granted notification includes the updated due date. The event uses `extension_assessed.html.erb` and `extension_assessed.text.erb`. | +| Where it is raised | `app/models/comments/extension_comment.rb`, in `ExtensionComment#assess_extension`, after `save!`. | + +## Failure behaviour + +No notification or email is sent when: + +- the extension request was already assessed; +- the deadline prevents an extension from being granted; or +- `Task#grant_extension` does not successfully apply the extension. + +A failed `grant_extension` call also leaves the extension request unassessed so that the system does not record or communicate a false successful result. + +## Tests + +`test/models/notification_extension_test.rb` + +The tests cover: + +- granted extensions and the updated due date; +- rejected extensions; +- already-assessed requests; +- deadline failures; +- a failed `grant_extension` operation; +- HTML and text event-specific templates. \ No newline at end of file diff --git a/docs/notifications/events/extension_requested.md b/docs/notifications/events/extension_requested.md new file mode 100644 index 0000000000..9c3feb4f75 --- /dev/null +++ b/docs/notifications/events/extension_requested.md @@ -0,0 +1,59 @@ +# Event: extension_requested + +| Field | Value | +|---|---| +| Event name | `extension_requested` | +| Category | `task` | +| What triggers it | A student asks for an extension. `Task#apply_for_extension` saves the `ExtensionComment`, runs any automatic approval, then calls `notify_extension_request_recipient`. | +| Who receives it | `extension.recipient`, set by `apply_for_extension` and read rather than worked out again. That is `Task#tutor`, which is `Project#tutor_for` and so the unit's main convenor when the student has no tutor. When more weeks are asked for than are left before the due date (`Task#weeks_can_extend`) it is the main convenor directly, though the request endpoint caps the weeks at that number so only a direct model call reaches that branch. A blank recipient is guarded. | +| Preference that gates it | The recipient's `receive_task_notifications`, shown to staff as "Task notifications" in their profile. Turned off, nothing is created, emailed or pushed. | +| Email subject | `#{product name}: New notification`, built by `NotificationsMailer#single_notification`, the same generic subject as every other event. | +| Email body summary | Names the student and task and asks the reader to open the task to grant or reject the request. The reason the student gave is left out. It ends with the same link to manage notification preferences as `task_submitted`. Templates are `app/views/notifications_mailer/extension_requested.text.erb` and `.html.erb`. | +| Where it is raised | `app/models/task.rb`, in `Task#notify_extension_request_recipient`, called at the end of `Task#apply_for_extension`, which `POST /projects/:project_id/task_def_id/:task_definition_id/request_extension` calls. | + +## Guards + +Only a request still waiting on a person notifies. + +- The requester must be the project's student. Staff creating an extension are + assessed on the spot and never notify, even when that assessment fails. +- A request the unit approves automatically is assessed before the notification + is considered, so it does not notify. The student hears the outcome through + `extension_assessed`. +- The check is `extension.assessed?`, not the automatic approval setting. An + automatic approval that cannot be applied leaves the request waiting, and + that one does notify. +- The recipient is never the person who asked. + +The message is `"#{user.name} asked for an extension on #{task_definition.name} in #{product_name}."` +and the link is `/projects//dashboard/`, the same +shape `task_submitted` uses. The notification targets the extension comment, so +opening the task's comments clears it from the bell. The comment itself stays +unread for the recipient until someone assesses it, as it did before. + +## Why it is a task notification + +`extension` has no entry in `Notification::PREFERENCE_FOR_TYPE`, so an +`extension` notification is always sent and a tutor could not switch it off. +As `task` it sits under "Task notifications" with `task_submitted` and +`task_help_requested`, the other things a student asks a tutor to act on. The +event name still says what happened. The student-facing `extension_assessed` +keeps the `extension` category. + +The web app has no presentation entry for this event yet, so it falls back to +the category and shows the generic task icon and "Task" label rather than the +extension icon. Adding `extension_requested` to `EVENT_PRESENTATIONS` in +`notification-presentation.ts` would change that. + +## Privacy + +The in-app message and email name the student and task, and leave out the +reason the student gave. The push body is the fixed +`A student asked for an extension.` from +`PushNotificationService::LOCK_SCREEN_BODY_OVERRIDES`. That copy follows the +MN-S04 rule but has not been added to the MN-S04 sign-off in +`docs/notifications/reviews/`. + +## Tests + +`test/models/notification_extension_request_test.rb` diff --git a/docs/notifications/events/group_membership_changed.md b/docs/notifications/events/group_membership_changed.md new file mode 100644 index 0000000000..19c9827764 --- /dev/null +++ b/docs/notifications/events/group_membership_changed.md @@ -0,0 +1,62 @@ +# Event: group_membership_changed + +| Field | Value | +|---|---| +| Event name | `group_membership_changed` | +| Category | `general` | +| What triggers it | A student's direct group membership changes through `Group#add_member` or `Group#remove_member`. Internal tutorial-switch operations and bulk CSV imports are intentionally suppressed. | +| Who receives it | Only the student whose membership changed (`project.student`). Other members of the group are not notified. This recipient scope was confirmed with the Email Notifications lead. | +| Preference that gates it | none, always sent | +| Email subject | `#{product name}: New notification`, using the existing `NotificationsMailer#single_notification` subject | +| Email body summary | Tells the affected student that they were added to or removed from a group. The membership change is not broadcast to other group members. Templates are `app/views/notifications_mailer/group_membership_changed.text.erb` and `group_membership_changed.html.erb` | +| Where it is raised | `app/models/group.rb:154` in `Group#add_member` and `app/models/group.rb:175` in `Group#remove_member`. Both call the private `notify_group_membership_change` helper at line 187 | + +## Recipient scope + +The agreed scope is **student only**. + +Only the student who was added to or removed from the group receives the notification. Other group members are not notified because a removal should not be broadcast to the group, and notifying the whole group would multiply the number of sends for a single membership change. + +## Tutorial switch guard + +`Group#switch_to_tutorial` temporarily removes and re-adds members while moving the group to another tutorial. These internal membership operations do not represent a real group membership change and must not send a leave-then-join notification pair. + +## Bulk CSV import guard + +`Unit#import_student_groups_from_csv` may add many students in one request. It calls `Group#add_member(..., notify: false)` so the import does not create and deliver one notification for every CSV row. + +If bulk-import notifications are required later, they should be queued or batched after a successful import rather than delivered separately inside the import request. + +## Implementation + +The event uses: + +- `type: 'general'` +- `event: 'group_membership_changed'` +- recipient: `project.student` +- `link: "/projects/#{project.id}/groups"` + +Event-specific HTML and text email templates are provided under `app/views/notifications_mailer/`. + +## How to check it by hand + +1. Use a unit with Group Work enabled and an existing student project. +2. Add the student to a group. +3. Open Mailpit at `http://localhost:8025` and confirm that one email is sent to the affected student. +4. Remove the same student from the group and confirm that one removal email is sent. +5. Confirm that no other group members receive the notification. +6. Move the group to another tutorial and confirm that the temporary remove/add operations do not create a leave-then-join email pair. + +## Tests + +`test/models/notification_group_test.rb` + +The tests cover: + +- adding a member sends one notification to the affected student +- removing a member sends one notification to the affected student +- other group members are not notified +- `switch_to_tutorial` does not send a leave-then-join notification pair +- a notification failure does not stop the membership change +- bulk CSV imports add students without raising per-student notifications +- the push payload points to the affected project's group page diff --git a/docs/notifications/events/new_task_available.md b/docs/notifications/events/new_task_available.md new file mode 100644 index 0000000000..ae699a1644 --- /dev/null +++ b/docs/notifications/events/new_task_available.md @@ -0,0 +1,139 @@ +# New Task Available Notification + +## Event + +`new_task_available` + +## Purpose + +Notifies eligible students when a newly created task becomes available. + +## Trigger + +The notification fan-out is queued after a task definition is successfully +created through any supported workflow: + +- the normal task-definition API; +- CSV task import; or +- task copying during unit rollover. + +These workflows enqueue only after the task definition is fully populated. A +general `TaskDefinition` `after_create` callback is intentionally not used +because CSV import and rollover save intermediate records before their full +workflow is complete. + +`SendNewTaskAvailableNotificationsJob` also checks active units once a day. +It sends on or shortly after the student's effective start date, covering task, +target-grade and student-specific future dates without creating missing `Task` +rows. A seven-day catch-up window tolerates short worker outages and late +enrolment without announcing historical tasks. + +A tracking timestamp marks definitions that are ready for this sweep. New +application code leaves the marker empty until an explicit workflow completes; +the database supplies a UTC marker only for writes from an older process during +a rolling deployment. The sweep gives those compatibility rows an hour to +settle before evaluating them. + +## Notification + +- Type: `task` +- Event: `new_task_available` +- Recipient: eligible students enrolled in the unit +- Preference: `receive_task_notifications` + +`NotificationService` applies the existing task-notification preference before creating and delivering the notification. + +## Recipient eligibility + +A student receives the notification only when all of the following are true: + +- The task was created through a supported direct, copy/rollover or import workflow. +- The unit is active. +- The student is currently enrolled in the unit. +- The task applies to the student's target grade. +- The student's effective task start date is now or earlier for an immediate + creation fan-out, or within the scheduled release check's bounded window. +- The student has task notifications enabled. + +The student's effective start date is determined with +`Webcal.start_date_for_task_definition`, the same calculation used by the +student calendar. Flexible dates, target-grade dates and supported +student-specific date adjustments are therefore respected. + +## Fan-out + +Notification delivery is not performed directly inside the API request. + +After a task-definition creation workflow completes, it enqueues: + +`NewTaskAvailableNotificationJob` + +The job processes enrolled projects in batches and sends one notification to +each eligible student whose task is already available. The scheduled release +job processes future start dates in daily batches. + +Duplicate notifications for the same student and task are prevented by an +immutable task-definition key backed by a unique database index. Renaming a +task does not resend it, while a genuinely new definition that reuses an old +abbreviation can still notify. Delivery is serialized on the notification row, +outside the student's project lock, so normal retries do not duplicate a +completed delivery. External email and push remain at-least-once: a process +failure after a provider accepts a message but before completion is recorded +can repeat that external message on retry. + +## Email templates + +HTML: + +`app/views/notifications_mailer/new_task_available.html.erb` + +Plain text: + +`app/views/notifications_mailer/new_task_available.text.erb` + +## Link + +The notification links the student to the new task on their project dashboard: + +`/projects/:project_id/dashboard/:task_abbreviation` + +## Future and bulk-created tasks + +Future-dated tasks are not announced early. The scheduled release job notifies +each student on or shortly after their effective start date. Unit rollover and +CSV import enqueue only after their multi-step workflow completes, so a worker +cannot observe a partially populated task definition. Updated CSV rows do not +generate a new-task notification. + +All paths use the same event, preference gate and duplicate guard. A bulk import +may deliberately enqueue one cohort fan-out per newly created task, but none of +that email delivery happens in the API request. + +## Tests + +Automated tests are located at: + +`test/models/notification_new_task_test.rb` + +The tests cover: + +- notification for an eligible student +- fan-out to multiple eligible students +- task-notification preference disabled +- unenrolled students +- task target-grade eligibility +- inactive units +- future base, target-grade and student-specific start dates +- rollover/copy and new CSV-import rows +- no notification for updated CSV rows +- no `Task` rows created by the scheduled sweep +- immutable database deduplication and retry state +- rolling-deployment tracking compatibility + +Test command: + +`bundle exec rails test test/models/notification_new_task_test.rb` + +The focused tests are also in: + +`test/sidekiq/send_new_task_available_notifications_job_test.rb` diff --git a/docs/notifications/events/portfolio_received.md b/docs/notifications/events/portfolio_received.md new file mode 100644 index 0000000000..dd50f21012 --- /dev/null +++ b/docs/notifications/events/portfolio_received.md @@ -0,0 +1,87 @@ +# Event: portfolio_received + +| Field | Value | +|---|---| +| Event name | `portfolio_received` | +| Category | `portfolio` | +| What triggers it | A student successfully starts a new manual portfolio submission through `PUT /projects/:id` with `compile_portfolio: true`. A repeated request while the same manual submission is already pending is not a new submission. | +| Who receives it | The submitting student (`project.student`). | +| Preference that gates it | `receive_portfolio_notifications` | +| Email subject | `#{product name}: New notification`, using the existing `NotificationsMailer#single_notification` subject | +| Email body summary | Confirms the date and time that the portfolio submission was received, including its timezone and UTC offset, and links to the project's current status. It says explicitly that the receipt does not confirm an assessment outcome. No portfolio contents, marks, grades, feedback, or other student information are included. Templates are `app/views/notifications_mailer/portfolio_received.text.erb` and `portfolio_received.html.erb`. | +| Where it is raised | `app/api/projects_api.rb`, in the `PUT /projects/:id` `compile_portfolio` branch, after the new submission state and `portfolio_submission_date` have been saved. | + +## Existing portfolio emails are different events + +The existing email audit records `PortfolioEvidenceMailer#portfolio_ready` and +`PortfolioEvidenceMailer#portfolio_failed`. Those messages are raised later by +`submission:generate_pdfs` after portfolio generation succeeds or fails. + +`portfolio_received` is the earlier receipt for accepting the student's +submission. It uses the shared `NotificationService` email path and does not +call either legacy portfolio mailer. One accepted submission therefore sends +one receipt, while a later generation result remains a separate event. + +## New-submission guard + +A receipt is raised only when a new manual submission is accepted: + +- `compile_portfolio` changes from false to true; or +- a pending auto-generated portfolio is replaced by the student's manual + submission. + +Retrying `compile_portfolio: true` while the same manual submission is already +pending does not create another notification and does not replace the original +`portfolio_submission_date`. Setting `compile_portfolio: false` does not send a +receipt. Once generation has finished and the flag is false again, a later +manual resubmission is new and receives its own receipt. + +The decision and save happen while holding a row lock on the project, so two +concurrent retries cannot both observe the submission as new. + +## Receipt time + +The saved `project.portfolio_submission_date` is the source of truth. It is +rendered in the project's campus timezone. When the project has no campus +timezone, the application timezone is used. The email includes the local date, +time, timezone abbreviation and numeric UTC offset, for example: + +`23 August 2026 at 10:34 PM AEST (UTC+10:00)` + +This is receipt metadata only. The notification and email never include the +portfolio, an assessment result, marks, grades, feedback or rationale. + +## Implementation + +The event uses: + +- `type: 'portfolio'` +- `event: 'portfolio_received'` +- recipient: `project.student` +- `link: "/projects/#{project.id}/dashboard"` + +`NotificationService` applies `receive_portfolio_notifications` before it +creates the in-app notification or delivers email and push. A failure while +raising the notification is logged without rejecting the already accepted +portfolio submission. + +## How to check it by hand + +1. Sign in as a student and submit a portfolio. +2. Confirm that exactly one receipt appears in Mailpit and that it is addressed + to the submitting student. +3. Confirm that the receipt contains the saved date, time, timezone and UTC + offset, and contains no portfolio or assessment content. +4. Repeat the same request while generation is pending and confirm that no + second receipt appears. +5. Turn off portfolio notifications, submit again after generation has + completed, and confirm that no notification or email is created. + +## Tests + +`test/models/notification_portfolio_test.rb` + +The focused tests cover the recipient, event/type, generic subject, push link, +event-specific copy, timestamp and privacy boundary, portfolio preference, +duplicate-request guard, later resubmission, manual replacement of an +auto-generated portfolio, cancellation, and notification failure isolation. diff --git a/docs/notifications/events/portfolio_submitted.md b/docs/notifications/events/portfolio_submitted.md new file mode 100644 index 0000000000..bc52ee8919 --- /dev/null +++ b/docs/notifications/events/portfolio_submitted.md @@ -0,0 +1,33 @@ +# Event: portfolio_submitted + +| Field | Value | +|---|---| +| Event name | `portfolio_submitted` | +| Category | `portfolio` | +| What triggers it | A new manual portfolio submission through `PUT /api/projects/:id` with `compile_portfolio: true`, the same moment `portfolio_received` goes to the student. | +| Who receives it | Every tutor of the student's tutorial enrolments. With none, the unit's main convenor. The student and the staff member who made the request are skipped. | +| Preference that gates it | The recipient's `receive_portfolio_notifications`. | +| Email subject | `#{product name}: New notification`, the generic subject. | +| Email body summary | Names the student and unit, with one button to the staff portfolio view (`Notification#web_path`). Templates are `app/views/notifications_mailer/portfolio_submitted.html.erb` and `.text.erb`, on the shared `notification_mail` layout. | +| Where it is raised | `app/api/projects_api.rb`, in the `notify_portfolio_submitted` helper. | + +## Guards + +- only a new submission notifies, the same test `portfolio_received` uses; +- the dedupe key is the project id plus the submission time, so a retried + request does not notify twice and a later resubmission does; +- failures are logged and never stop the submission. + +The message is `"#{student.name} submitted a portfolio in #{unit.code}."`, the +link is `/projects//dashboard` and the notifiable is the project, so +the web client can open the staff portfolio view from `unit_id` and +`project_id`. + +## Privacy + +The push body is the fixed `A student submitted a portfolio.` so the student's +name does not reach a lock screen. + +## Tests + +`test/models/notification_portfolio_submitted_test.rb` diff --git a/docs/notifications/events/task_comment_created.md b/docs/notifications/events/task_comment_created.md new file mode 100644 index 0000000000..cd2b208474 --- /dev/null +++ b/docs/notifications/events/task_comment_created.md @@ -0,0 +1,81 @@ +# Event: task_comment_created + +The first notification event wired into OnTrack. Ticket EN-E01. + +This is the worked example. If you are adding an event, copy the shape of this +one. + +## What it does + +Someone posts a text comment on a task. The other party is told. + +- A tutor comments, the student is emailed. +- A student comments, the tutor is emailed. + +## Where it is raised + +`app/models/task.rb`, in `notify_comment_recipient`, called at the end of +`add_text_comment` once the comment has saved. + + NotificationService.notify( + user: comment.recipient, + type: 'feedback', + event: 'task_comment_created', + message: "#{comment.user.name} commented on #{task_definition.abbreviation} in #{unit.code}.", + link: "/projects/#{project.id}/dashboard/#{task_definition.abbreviation}" + ) + +## Fields + +| Field | Value | +|---|---| +| `type` | `feedback`, so the recipient's `receive_feedback_notifications` switch controls it | +| `event` | `task_comment_created` | +| `message` | Who commented, which task, which unit. Never the comment text | +| `link` | `/projects//dashboard/` | + +## Templates + +- `app/views/notifications_mailer/task_comment_created.text.erb` +- `app/views/notifications_mailer/task_comment_created.html.erb` + +`NotificationsMailer#single_notification` picks the template named after the +event when it exists, and falls back to `single_notification.*.erb` when it does +not. That is why adding an event never requires editing the mailer. + +## Three things to know before you copy this + +1. **Do not work out who to notify.** `comment.recipient` is already set by + `add_text_comment`: the tutor when a student commented, the student when a + tutor commented. + +2. **Guard for no recipient.** A project with no tutor for the task definition + has no recipient. `notify_comment_recipient` returns early. Without that it + raises. + +3. **A notification must never break the thing that triggered it.** The call is + wrapped in a `rescue StandardError` that logs and swallows. Posting a comment + must succeed even if notifying fails. + +## How to check it by hand + +1. Sign in as a tutor, open a student's task, post a comment. +2. A file appears in `doubtfire-deploy/data/tmp/mails/`, addressed to the + student. It names the commenter and the task, and does not contain the + comment text. +3. Turn that student's feedback notifications off in their profile, comment + again, and no new file appears. + +## Tests + +`test/models/notification_task_comment_test.rb` + +Covers both directions, the preference switch, the absent recipient, the comment +text staying out of the email, and that a notification failure still leaves the +comment saved. + +## Known limitation + +The email subject is the generic "New notification". Per-event subjects would +need a shared lookup in the mailer, which would make every event ticket edit the +same file and collide. Left as it is on purpose. diff --git a/docs/notifications/events/task_due_date_changed.md b/docs/notifications/events/task_due_date_changed.md new file mode 100644 index 0000000000..b2c19687ec --- /dev/null +++ b/docs/notifications/events/task_due_date_changed.md @@ -0,0 +1,101 @@ +# Event: task_due_date_changed + +## Purpose + +Notify eligible students when a convenor changes a task definition's due date +through the normal task-definition update API. + +## Trigger + +The trigger is in `app/api/task_definitions_api.rb`. + +Immediately after `task_def.update!(task_params)`, the API captures +`saved_change_to_due_date`. After the rest of the update succeeds, it enqueues +`TaskDueDateChangedNotificationJob`. + +A `TaskDefinition` model callback is deliberately not used. Task definitions +can also be saved by unit date propagation, imports, rollovers, copies and +internal maintenance. A model callback could therefore create unexpected +cohort-wide email fan-out. + +## Queue + +The API request does not perform the cohort email and push fan-out directly. +`TaskDueDateChangedNotificationJob` performs the fan-out through Sidekiq. + +A functioning Sidekiq worker must consume the same +`DF_REDIS_SIDEKIQ_URL` used by the API. + +## Recipient eligibility + +A project is eligible only when: + +- the unit is active; +- the project is enrolled; +- the project's target grade is at least the task definition's target grade; + and +- the student has task notifications enabled. + +Recipients are selected from projects rather than existing Task rows. OnTrack +creates Task rows on demand, so an eligible student may not yet have one. The +notification job does not create Task rows. + +## Stale jobs and duplicate queue entries + +The job receives: + +- the task definition ID; +- the previous stored due date; and +- the new stored due date. + +Before sending, it checks that the task definition still has the queued new raw +due date. This prevents an outdated job from sending after the due date has +changed again. + +Sidekiq uniqueness rejects another pending or executing job with the same task +definition ID and old/new date values. + +Automatic retries are disabled because retrying a partly completed cohort +fan-out could create duplicate notifications for students already processed. +A failure for one project is logged without stopping the remaining projects. + +## Notification fields + +- Type: `task` +- Event: `task_due_date_changed` +- Message: names the task and unit but does not expose the due date +- Link: `/projects/:project_id/dashboard/:task_abbreviation` +- Preference: `receive_task_notifications` + +## Bulk unit date changes + +Changing a unit start date updates many task definitions internally. This +implementation does not send one email per changed task for that path. + +A future unit-level notification or digest should cover bulk schedule changes +without sending many separate emails to each student. + +## Templates + +- `app/views/notifications_mailer/task_due_date_changed.text.erb` +- `app/views/notifications_mailer/task_due_date_changed.html.erb` + +## Tests + +- `test/sidekiq/task_due_date_changed_notification_job_test.rb` +- `test/api/units/task_definitions_api_test.rb` + +The tests cover: + +- eligible students without Task rows; +- target-grade filtering; +- withdrawn students; +- notification preferences; +- inactive units; +- stale jobs; +- privacy-safe message content and links; +- the event-specific email template; +- enqueue on a due-date API update; +- no enqueue for unrelated updates; +- no enqueue from direct model updates; and +- queue failure not breaking the core due-date update. \ No newline at end of file diff --git a/docs/notifications/events/task_due_soon.md b/docs/notifications/events/task_due_soon.md new file mode 100644 index 0000000000..ceea92f707 --- /dev/null +++ b/docs/notifications/events/task_due_soon.md @@ -0,0 +1,175 @@ +# Event: task_due_soon + +## Purpose + +Remind a student that a task they still owe is nearly due, so a deadline is not +the first they hear about it. + +## Trigger + +Nothing a person does. Every other event in this folder hangs off an action: +a comment is posted, a due date is edited, a status changes. A deadline getting +closer is nobody doing anything, so there is no model to hook and no callback to +add. The reminder has to be swept for. + +`SendDueSoonRemindersJob` does the sweep. `config/schedule.yml` runs it under the +name `send_due_soon_reminders`. + +## Schedule + +`every day at 8am`. + +Once a day, because a deadline only moves once a day and every notification this +job raises sends an email. Sweeping every thirty minutes would find the same +answer forty-eight times. + +At a fixed time, because that also fixes when the emails land. On a short +interval a student gets theirs at whatever hour their task happened to cross +into the window, which is 3am as often as any other hour. Pinning it to the +morning makes the reminder arrive on a day somebody can act on it. + +Missing a run costs nothing. The job asks whether a task is due within the next +three days, not whether it crossed a line since the last run, so tomorrow's run +still catches everything a skipped run would have. + +**Which 8am depends on the environment.** There is no timezone in the cron +expression, no `config.time_zone` in the app, and sidekiq-cron reads the process +clock, so the hour comes from `TZ`. `development/api.env` sets +`TZ=Australia/Melbourne`, which is what makes this a morning. A deployment that +leaves `TZ` unset gets the image default of UTC and sends these in the evening +local time. `aggregate_task_completion_stats`, at 11:55pm, already depends on the +same thing. + +**This has not been seen to fire.** There is no Sidekiq worker in the dev stack, +which is EN-F03, so the schedule entry is unverified beyond the assertion in +`test/sidekiq/scheduled_job_test.rb` that it loads and enqueues. + +## Window + +Three days, `SendDueSoonRemindersJob::WINDOW_DAYS`. + +Long enough to still do something about it over a weekend, short enough that the +reminder is about this task and not about the rest of the trimester. +`Project#top_tasks` calls seven days "soon" for its own purposes, and seven days +of warning on a weekly task is most of the tasks a student has, which is a list +rather than a reminder. + +## Recipient eligibility + +A project gets a reminder about a task definition only when: + +- the unit is active; +- the project is enrolled; +- the project has a target grade set; +- the task definition is assigned at that target grade; +- the task's status is one of `not_started`, `working_on_it`, `need_help`, + `fix_and_resubmit` or `redo`; and +- the student has task notifications enabled, which `NotificationService` + enforces rather than this job. + +Recipients come from projects and not from Task rows. OnTrack creates a Task row +the first time anyone touches the task, so the students who have not started +have no row, and they are the ones a reminder is for. + +**Nothing in this job may call `Project#task_for_task_definition`**, which +creates the row it cannot find, and nothing may call +`Project#task_definitions_and_status` either, because that calls it. The job +reads `project.tasks` once per project and looks each definition up in a hash. + +That is also why it is affordable. `task_definitions_and_status` asks for the +assigned definitions per project and then runs two more queries per definition, +so a five hundred student unit with twenty task definitions is upwards of twenty +thousand queries. Here the definitions are read once per unit and the tasks once +per project. + +### Statuses deliberately left out + +`discuss` and `demonstrate` both mean the student has submitted and is waiting on +a tutor. Telling them the task is due soon is wrong, and it is the kind of wrong +that teaches people to stop reading notifications, so `OUTSTANDING_STATUSES` +lists only the five that mean work is still owed. A project with no Task row at +all counts as outstanding. + +## Which deadline + +Per student, not per task definition, and answered by +`Webcal.end_date_for_task_definition(task_definition, task, project)`. + +That method already exists, already handles both cases, and is what the +student's calendar feed shows them, so writing the rule again here would mean +two answers to "when is this due" that could disagree. + +- With a Task row, `Task#local_due_date`, which knows about extensions and about + a unit's flexible dates. +- Without one, the grade level override when the unit has flexible dates, and + the task definition's own `target_date` otherwise. + +The second case is the one that is easy to get wrong. On a unit with flexible +dates the grade override applies before any Task row exists, so a grade 2 +student can be due days away from the unit's own date without ever having opened +the task. `Project#top_tasks` reads `target_date` directly and has this gap; +`Webcal` does not, which is why this follows `Webcal`. + +## Duplicates + +One reminder per student per task, ever. Before notifying, the job asks: + +```ruby +Notification.exists?(user_id:, notification_type: 'task', event: 'task_due_soon', link:) +``` + +The index `index_notifications_on_user_id_and_event` is what makes that cheap +enough to ask once per candidate task. Without the guard the job runs again +tomorrow, the task is still due soon tomorrow, and the same student is emailed +every morning until the deadline passes. + +Known consequence: a task whose deadline is later extended past the window and +then comes back into it does not produce a second reminder. That is deliberate. +The student asked for the extension, so they know about the task, and +`task_due_date_changed` covers a convenor moving it. + +## Notification fields + +- Type: `task` +- Event: `task_due_soon` +- Message: `" in is due soon."` +- Link: `/projects/:project_id/dashboard/:task_abbreviation` +- Preference: `receive_task_notifications` + +The date stays out of the message, the same as `task_due_date_changed`. The row +outlives the deadline it describes, so "due on the 14th" is wrong a week later +while "due soon" only stops being interesting. + +## Failure handling + +A failure on one project is logged, its id is collected, and the sweep carries on +through the rest of the cohort. At the end of `perform` the collected ids are +raised, which is what `NewTaskAvailableNotificationJob` does and for the same +reason: logging and returning normally would leave `perform` successful, Sidekiq +would schedule no retry, and a student whose task is due today is filtered out as +overdue tomorrow, so that reminder is gone for good. + +`retry: 1`. Re-running the whole sweep is only safe because of the duplicate +guard, which skips everyone already reminded. + +Sidekiq uniqueness rejects a second copy of the sweep, +`lock: :until_executed` on a fixed lock argument. + +## Templates + +- `app/views/notifications_mailer/task_due_soon.text.erb` +- `app/views/notifications_mailer/task_due_soon.html.erb` + +## Tests + +`test/sidekiq/send_due_soon_reminders_job_test.rb`, seventeen cases, covering +eligible students without Task rows, both edges of the window, an already passed +deadline, the duplicate guard, withdrawn students, inactive units, the +notification preference, target grade filtering, a student's own extended +deadline, a flexible unit's grade level deadline with no Task row, a task +waiting on a tutor, a failure being raised rather than swallowed, privacy-safe +content, and the event template. + +`test/sidekiq/scheduled_job_test.rb` covers the schedule entry loading and +enqueuing. It counts the entries in `config/schedule.yml`, so adding this one +changed the expected count from six to seven. diff --git a/docs/notifications/events/task_help_requested.md b/docs/notifications/events/task_help_requested.md new file mode 100644 index 0000000000..a67addf29f --- /dev/null +++ b/docs/notifications/events/task_help_requested.md @@ -0,0 +1,43 @@ +# Event: task_help_requested + +| Field | Value | +|---|---| +| Event name | `task_help_requested` | +| Category | `task` | +| What triggers it | A student moves their task to Need Help through `Task#trigger_transition`, and the status actually changes. That is either the status menu, or uploading work with `trigger: 'need_help'`, which `Task#create_submission_and_trigger_state_change` turns into the same transition. | +| Who receives it | `project.tutor_for(task_definition)`, the task's tutor, or the unit's main convenor when the student has no tutor for that task. A blank recipient is guarded and raises nothing. | +| Preference that gates it | The recipient's `receive_task_notifications`, shown to staff as "Task notifications" in their profile. | +| Email subject | `#{product name}: New notification`, built by `NotificationsMailer#single_notification`, the same generic subject as every other event. | +| Email body summary | Names the student and task so the tutor knows who is stuck, with a link to the task. The student's work and comments are left out. Templates are `app/views/notifications_mailer/task_help_requested.text.erb` and `.html.erb`. | +| Where it is raised | `app/models/task.rb`, in `Task#notify_tutor_of_student_request`, called at the end of `Task#trigger_transition`. The same method raises `task_submitted`. | + +## Guards + +It shares every guard with `task_submitted`, because it is the same method: + +- only a student or group member moving the task notifies, so a tutor setting + Need Help raises `task_status_changed` for the student and nothing for the + tutor; +- an unchanged status does not notify again; +- calls marked `group_transition: true` are suppressed. Changing the status + of a group task does not spread Need Help to the other members, but + uploading group work with the Need Help trigger does, once per member task. + Only the uploader's own task notifies, so the tutor hears about it once; +- the tutor is never notified about their own action. + +The message is `"#{student.name} asked for help with #{task_definition.name} in #{product_name}."` +and the link is `/projects//dashboard/`, the same +shape `task_submitted` uses. The notification targets the task, so opening the +task's comments clears it from the tutor's bell. + +## Privacy + +The in-app message and email name the student and task. The push body is the +fixed `A student asked for help with a task.` from +`PushNotificationService::LOCK_SCREEN_BODY_OVERRIDES`, so neither reaches a lock +screen. That copy follows the MN-S04 rule but has not been added to the MN-S04 +sign-off in `docs/notifications/reviews/`. + +## Tests + +`test/models/notification_task_help_requested_test.rb` diff --git a/docs/notifications/events/task_status_changed.md b/docs/notifications/events/task_status_changed.md new file mode 100644 index 0000000000..a579342373 --- /dev/null +++ b/docs/notifications/events/task_status_changed.md @@ -0,0 +1,108 @@ +# Event: task_status_changed + +A staff member changes the status of a task. The student is told. Ticket EN-E02. + +Built the same way as the worked example in `task_comment_created.md`; read that +one first. + +## What it does + +A tutor marks a task, and the student whose task it is gets an email telling them +the status changed. + +- A tutor changes the status, the student is emailed. +- A student changing their own task is not emailed about their own action. + +## Where it is raised + +`app/models/task.rb`, in `notify_student_of_status_change`, called at the end of +`trigger_transition` once the transition has succeeded and the new status has +been saved. + + NotificationService.notify( + user: project.student, + type: 'task', + event: 'task_status_changed', + message: "#{by_user.name} updated the status of #{task_definition.abbreviation} in #{unit.code}.", + link: "/projects/#{project.id}/dashboard/#{task_definition.abbreviation}" + ) + +## Fields + +| Field | Value | +|---|---| +| `type` | `task`, so the student's `receive_task_notifications` switch controls it | +| `event` | `task_status_changed` | +| `message` | Who acted, which task, which unit. Never the new status value | +| `link` | `/projects//dashboard/` | + +## Templates + +- `app/views/notifications_mailer/task_status_changed.text.erb` +- `app/views/notifications_mailer/task_status_changed.html.erb` + +`NotificationsMailer#single_notification` picks the template named after the +event when it exists. Adding this event never required editing the mailer. + +## Three things to know before you copy this + +1. **Only a staff action notifies.** The guard is `role == :tutor`. A student + changing their own task (submitting, working on it) must never email + themselves. `role` is already worked out at the top of `trigger_transition`. + +2. **Only a real change notifies.** The status before the transition is captured + and compared at the end. Re-applying the same status is a no-op and sends + nothing. + +3. **A notification must never break the transition.** The call is wrapped in a + `rescue StandardError` that logs and swallows. Marking a task must succeed + even if notifying fails. + +## How to check it by hand + +1. Sign in as a tutor, open a student's task, change its status. +2. An email to the student arrives at http://localhost:8025 (Mailpit). It names + the tutor and the task, and does not contain the new status value. +3. Sign in as that student, change one of their own tasks, and confirm no email + is sent to themselves. +4. Turn that student's task notifications off in their profile, have the tutor + mark again, and no email arrives. + +## Known limitations and deliberate choices + +- **Bulk marking queues one email and one push job per task.** `Project#trigger_week_end` + (`app/models/project.rb`) loops `trigger_transition(trigger: 'complete', + bulk: true)` over a student's discuss/demonstrate tasks, and this event ignores + the `bulk:` flag, so a single request can enqueue several near-identical + notifications. Provider delivery occurs in Sidekiq, but the queue hand-offs + still happen in that request. The path is latent today — no + `doubtfire-web` caller drives `trigger_week_end`. Left un-suppressed on purpose + so bulk-marked tasks still notify; batching many into one email belongs with the + event design, not here. + +- **The fix-and-resubmit cascade does not raise this event.** Inside `assess`, the + `recursive_fix` cascade calls `assess` directly on dependent tasks instead of + going through `trigger_transition`, so those status changes raise nothing here. + The student is still emailed, but via `task_comment_created` (the cascade adds an + automated comment) — i.e. under a different event. Reconciling that is out of + scope for EN-E02. + +- **Site admins acting on the student-side branches notify nobody.** `user_role` + returns `:admin` for an unenrolled site admin, who can still drive the + `working_on_it` / `need_help` / `not_started` / `ready_for_feedback` branches. + The `role == :tutor` guard means those changes notify no one. That is intended: + an admin poking at a task is not a tutor marking it for the student. + +## Tests + +`test/models/notification_task_status_test.rb` + +Covers the staff change, the student's own action sending nothing, an unchanged +status sending nothing, the preference switch, the status value staying out of +the email, the event-specific template being used, a bulk mark still notifying, +and that a notification failure still leaves the transition committed. + +Not yet covered: the group-task fan-out (each member emailed about their own +task). The behaviour is correct — `propagate_transition` threads `by_user` +through a per-member `trigger_transition` — but a factory-built group task test +is a worthwhile follow-up. diff --git a/docs/notifications/events/task_submitted.md b/docs/notifications/events/task_submitted.md new file mode 100644 index 0000000000..ff00fbb1db --- /dev/null +++ b/docs/notifications/events/task_submitted.md @@ -0,0 +1,63 @@ +# Event: task_submitted + +| Field | Value | +|---|---| +| Event name | `task_submitted` | +| Category | `task` | +| What triggers it | A student submits a task and it genuinely moves into the ready-for-feedback (ready-for-marking) state through `Task#trigger_transition`. | +| Who receives it | `project.tutor_for(task_definition)`, the tutor responsible for that task. A missing tutor is guarded and raises no notification. | +| Preference that gates it | The recipient tutor's `receive_task_notifications` preference. | +| Email subject | `#{product name}: New notification`, built by `NotificationsMailer#single_notification`. The generic subject deliberately does not identify the student or task. | +| Email body summary | Names the student and task so the assigned tutor knows what is waiting. The submission and all assessment content are deliberately omitted. Templates are `app/views/notifications_mailer/task_submitted.text.erb` and `.html.erb`. | +| Where it is raised | `app/models/task.rb`, in `Task#notify_tutor_of_student_request`, called at the end of `Task#trigger_transition` after a successful status change. The same method raises `task_help_requested`. | + +## Transition and duplicate guards + +`task_submitted` shares the successful-transition seam used by EN-E02's +`task_status_changed`, but the two events have disjoint actor guards: + +- `task_submitted` only runs for a student or group member moving a task into + `TaskStatus.ready_for_feedback`; +- `task_status_changed` only runs for a tutor changing a student's task. + +The previous and current status IDs are compared, so submitting again while the +task is already ready for feedback does not send another email. + +Group submissions can invoke the transition once per member task and then +visit those tasks again through submission propagation. Calls marked +`group_transition: true` are suppressed, leaving the original action as the +single notification boundary. + +## Privacy + +The email names the student and task because the recipient is the assigned +tutor and needs both to identify the waiting work. It does not contain uploaded +work, marks, grades, feedback, comments, or any other assessment content. The +link opens the authenticated task page in OnTrack. + +## Volume + +Independent submissions still produce one immediate email each. A tutor with +many students may therefore receive many messages in a short period. A digest +would require queueing and aggregation work beyond this event; this +implementation prevents duplicate amplification but does not batch unrelated +submissions. + +## How to check it by hand + +1. Sign in as a student and submit a task for marking. +2. Confirm exactly one email arrives for the tutor returned by + `project.tutor_for(task_definition)`. +3. Confirm the email names the student and task, links to the task, and contains + no submission or assessment content. +4. Repeat with the tutor's task notifications disabled and confirm no + notification is created or delivered. + +## Tests + +`test/models/notification_task_submitted_test.rb` + +The focused tests cover the recipient and preference, exact event/category, +ready-for-marking status guard, EN-E02 separation, duplicate and group +propagation guards, HTML/text copy, task link, nil tutor, push payload shape, +and failure isolation. diff --git a/docs/notifications/events/tutorial_changed.md b/docs/notifications/events/tutorial_changed.md new file mode 100644 index 0000000000..c0647694f5 --- /dev/null +++ b/docs/notifications/events/tutorial_changed.md @@ -0,0 +1,55 @@ +# Event: tutorial_changed + +| Field | Value | +|---|---| +| Event name | `tutorial_changed` | +| Category | `general` | +| What triggers it | An existing tutorial enrolment is moved in place by `Project#enrol_in`. A first enrolment, selecting the same tutorial again, and the multiple-enrolment collapse path do not trigger it. | +| Who receives it | Only the affected project's student (`project.student`). Students in the old or new tutorial are not notified. | +| Preference that gates it | none, always sent | +| Email subject | `#{product name}: New notification`, using the existing `NotificationsMailer#single_notification` subject | +| Email body summary | Names the new tutorial and gives its meeting day and time. It deliberately omits the old tutorial and any other student's details. Templates are `app/views/notifications_mailer/tutorial_changed.text.erb` and `tutorial_changed.html.erb`. | +| Where it is raised | `app/models/project.rb`, in the existing-enrolment update branch of `Project#enrol_in`, through the private `notify_tutorial_changed` helper | + +## Recipient and trigger guards + +The notification is addressed directly to `project.student`. It is not fanned +out through either tutorial's enrolments, so one student's move creates one +notification and one email. + +`Project#enrol_in` has distinct paths for creating a first enrolment, collapsing +multiple stream enrolments into a single non-stream enrolment, and updating an +existing enrolment. Only the final path calls `notify_tutorial_changed`, after +the tutorial enrolment update succeeds. Selecting the current tutorial returns +before any of those paths and does not notify. + +## Notification fields + +- Type: `general` +- Event: `tutorial_changed` +- Recipient: `project.student` +- Message: names the new tutorial abbreviation, unit, meeting day and meeting time +- Link: `/projects/:project_id/dashboard` +- Preference: none; `general` has no entry in `Notification::PREFERENCE_FOR_TYPE` + +The previous tutorial is intentionally absent from the notification record, +email and push body. Marks, feedback, comments and other student names are not +included. + +Notification errors are logged and do not roll back a successful tutorial move. + +## How to check it by hand + +1. As a convenor, move one student from an existing tutorial to another. +2. Confirm that exactly one email is sent to that student. +3. Confirm that the email names the new tutorial and its day and time, and does + not name the old tutorial or any other student. +4. Add a student to their first tutorial and confirm that no email is sent. + +## Tests + +`test/models/notification_tutorial_test.rb` + +The tests cover the affected-student recipient, notification fields and push +link, both email formats, privacy-safe copy, first enrolment, the same-tutorial +no-op, the multiple-enrolment collapse path, and notification failure isolation. diff --git a/docs/notifications/events/unit_announcement_published.md b/docs/notifications/events/unit_announcement_published.md new file mode 100644 index 0000000000..3f3dadffd3 --- /dev/null +++ b/docs/notifications/events/unit_announcement_published.md @@ -0,0 +1,29 @@ +# Event: unit_announcement_published + +| Field | Value | +|---|---| +| Event name | `unit_announcement_published` | +| Category | `unit_hub` | +| What triggers it | An announcement becomes visible to its unit: created with a publication time that has arrived, a draft given one, an expired one given a later expiry, or a visible one pinned. A future publication time waits for a job scheduled at that time. `UnitAnnouncement` calls `UnitHub::Notifications.announcement_committed` after commit. | +| Who receives it | `UnitHub::Notifications.recipients`: users with an enrolled project in the unit and the unit's tutors and convenors, the unit active, never the author. | +| Preference that gates it | `receive_unit_hub_notifications` for the whole category, then `receive_unit_hub_email_notifications` and `receive_unit_hub_push_notifications` for those channels. Defaults on, off, off. | +| Email subject | `#{product name}: New notification`, the generic subject. | +| Email body summary | Heading, unit, title and a 280 character summary in the details block, one button to the Unit Hub, preferences footer. Templates `unit_announcement_published.html.erb` and `.text.erb`. | +| Where it is raised | `UnitAnnouncementNotificationJob#notify_published`. | + +## Guards + +- dedupe key `unit_announcement_published::published-`, or + `pinned-` for a pin, per recipient, so a retry or a second job sends + nothing twice; +- a job for a publication time that has since moved is ignored; +- an announcement created with a publication time over a day before it was + created is a backfill and is not announced; +- unpinning is not announced. + +Link `/unit-hub?unit=&announcement=`. Push body is the fixed +`There is a new announcement in your unit.` + +## Tests + +`test/models/notification_unit_announcement_test.rb` diff --git a/docs/notifications/events/unit_announcement_updated.md b/docs/notifications/events/unit_announcement_updated.md new file mode 100644 index 0000000000..f357fd5b3d --- /dev/null +++ b/docs/notifications/events/unit_announcement_updated.md @@ -0,0 +1,33 @@ +# Event: unit_announcement_updated + +| Field | Value | +|---|---| +| Event name | `unit_announcement_updated` | +| Category | `unit_hub` | +| What triggers it | A visible announcement's title or body changes in a way `UnitHub::Notifications.meaningful_edit?` counts. | +| Who receives it | The same recipients as `unit_announcement_published`. | +| Preference that gates it | As `unit_announcement_published`. | +| Email subject | `#{product name}: New notification`, the generic subject. | +| Email body summary | As the published email, with the current title and summary. Templates `unit_announcement_updated.html.erb` and `.text.erb`. | +| Where it is raised | `UnitAnnouncementNotificationJob#notify_updated`. | + +## What counts as meaningful + +Case, spacing and punctuation never count. Any changed word containing a +digit counts. One word swapped for another within two letters of it, where both +are four letters or more, is a typo, and so is a doubled word taken out. Any +other change to the words counts. + +## Guards + +- at most one per announcement per recipient in 30 minutes, and a publish in + that window counts, so an edit straight after posting sends nothing; +- dedupe key `unit_announcement_updated::`; +- one update job per announcement at a time, it reads the announcement as it is + when it runs. + +Push body is the fixed `An announcement in your unit was updated.` + +## Tests + +`test/models/notification_unit_announcement_test.rb` diff --git a/docs/notifications/events/unit_session_changed.md b/docs/notifications/events/unit_session_changed.md new file mode 100644 index 0000000000..bd635d0f0d --- /dev/null +++ b/docs/notifications/events/unit_session_changed.md @@ -0,0 +1,29 @@ +# Event: unit_session_changed + +| Field | Value | +|---|---| +| Event name | `unit_session_changed` | +| Category | `unit_hub` | +| What triggers it | A published session that has not finished changes its start, end, time zone, recurrence, location or join link, is cancelled (including through the delete endpoint), or is put back on. `UnitLearningSession` calls `UnitHub::Notifications.session_committed` after an update commits. | +| Who receives it | `UnitHub::Notifications.recipients`, never the author. | +| Preference that gates it | As `unit_announcement_published`. | +| Email subject | `#{product name}: New notification`, the generic subject. | +| Email body summary | A changed session gets the new unit, title, when, where and repeat details. A cancellation gets its own heading, a bold line saying it is cancelled and a status row, and never the join link. Templates `unit_session_changed.html.erb` and `.text.erb`. | +| Where it is raised | `UnitSessionNotificationJob#perform`. | + +## Guards + +- dedupe key `unit_session_changed::v`; +- a queued time change for a session cancelled since is dropped, the + cancellation has its own job; +- creating a session, a description edit, a draft and a past session send + nothing. + +Message copy is `" in <unit> has changed. Now: <when>, <where>."`, or +`"<title> in <unit> on <when> is cancelled."`, or for a weekly session +`"The weekly <title> sessions in <unit> are cancelled."` Push body is the fixed +`A session in your unit has changed.` + +## Tests + +`test/models/notification_unit_session_test.rb` diff --git a/docs/notifications/events/unit_session_starting_soon.md b/docs/notifications/events/unit_session_starting_soon.md new file mode 100644 index 0000000000..80e3130398 --- /dev/null +++ b/docs/notifications/events/unit_session_starting_soon.md @@ -0,0 +1,26 @@ +# Event: unit_session_starting_soon + +| Field | Value | +|---|---| +| Event name | `unit_session_starting_soon` | +| Category | `unit_hub` | +| What triggers it | `SendUnitSessionRemindersJob`, every five minutes from `config/schedule.yml`, finds occurrences of published, uncancelled sessions in active units starting within the next 30 minutes. | +| Who receives it | `UnitHub::Notifications.recipients` who also have `receive_unit_hub_session_reminders` on, never the author. | +| Preference that gates it | `receive_unit_hub_session_reminders` (default off) as the opt-in, then the category and channel columns as `unit_announcement_published`. | +| Email subject | `#{product name}: New notification`, the generic subject. | +| Email body summary | Unit, session, when and where for that occurrence, one button to the Unit Hub, and a footer about session reminders. Templates `unit_session_starting_soon.html.erb` and `.text.erb`. | +| Where it is raised | `SendUnitSessionRemindersJob#remind`. | + +## Guards + +- dedupe key `unit_session_starting_soon:<id>:<occurrence start epoch>`, so the + six runs that see one occurrence send one reminder, and each weekly + occurrence gets its own; +- a run after the start sends nothing. + +The opt-in covers every session in the user's units. There is no per-session +sign-up to hang it on. Push body is the fixed `A session in your unit starts soon.` + +## Tests + +`test/models/notification_unit_session_test.rb` diff --git a/docs/notifications/existing-emails.md b/docs/notifications/existing-emails.md new file mode 100644 index 0000000000..f545ad6902 --- /dev/null +++ b/docs/notifications/existing-emails.md @@ -0,0 +1,196 @@ +# Existing Email Audit + +This document records email behaviour that already exists in OnTrack and the +shared email-delivery paths used by the v2 notification work. Its purpose is to +prevent new notification events from duplicating existing email behaviour or +accidentally sending multiple messages for the same action. + +The audit covers both the API mailers and the existing communication subsystem +in the web application. + +## Audit snapshot + +This audit was checked against `feature/notifications` at commit +`09a61714425f12e1412da5b7a34f31f7ea5612dd` on 15 August 2026. + +The primary reference for each send site is its class and method name. The +linked line ranges are pinned to the audited commit so later code changes do not +make the references point to unrelated code. + +The audit can be reproduced with: + +```bash +git grep -nE '\.deliver(_now|_later)?([^[:alnum:]_]|$)' \ + 09a61714425f12e1412da5b7a34f31f7ea5612dd -- app lib \ + | grep -Ev 'PushNotificationService\.deliver|def (self\.)?deliver(_now|_later)?([^[:alnum:]_]|$)' +``` + +The command returned 21 direct email delivery call sites after the push +delivery call was excluded. Each result was manually checked to confirm that +it sends an email. + +## API email send sites + +A search of `doubtfire-api` for `.deliver`, `.deliver_now`, and +`.deliver_later` identified 21 real email send sites. Push notification service +calls and method definitions are not counted as email send sites. + +| Existing email | Trigger / stable send-site reference | Recipient | Preference / guard | +| --- | --- | --- | --- | +| Turnitin error log | [`TurnItIn.handle_tii_error`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/helpers/turn_it_in.rb#L72-L80) – a Turnitin request returns HTTP 403 | Configured administrator/error-log address | Operational path; no user preference; attempted only for a 403 error | +| Task PDF failed – queued converter | [`PortfolioEvidence.process_new_to_pdf`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/models/portfolio_evidence.rb#L31-L75) – queued task PDF conversion reports a failure | Project student | `receive_task_notifications` | +| Weekly student summary | [`Project#send_weekly_status_email`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/models/project.rb#L619-L636) – weekly project summary is generated | Project student | `receive_feedback_notifications`; a final summary is skipped when a portfolio already exists | +| Task feedback ready | [`Unit#update_task_status_from_csv`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/models/unit.rb#L2632-L2755) – batch CSV/ZIP marking import finishes feedback/PDF processing | Project student | `receive_feedback_notifications` | +| Weekly staff summary | [`UnitRole#send_weekly_status_email`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/models/unit_role.rb#L187-L193) – weekly staff summary is generated | Staff member represented by the unit role | `receive_feedback_notifications` | +| Single notification email | [`NotificationService.notify` and `deliver_email`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/services/notification_service.rb#L24-L68) – an event is created and delivered through the notification service | Notification user | `task`, `feedback`, and `portfolio` map to existing preferences; `extension` and `general` currently have no mapping and are allowed by default | +| Task PDF failed – submission job | [`AcceptSubmissionJob#perform`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/accept_submission_job.rb#L10-L55) – submitted task PDF conversion fails | Project student | `receive_task_notifications` | +| Submission processing error | [`AcceptSubmissionJob#perform`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/accept_submission_job.rb#L10-L55) – submission processing raises an exception | Configured administrator/error recipient | Operational path; no user preference; only sent when an error mail is available | +| Archive error | [`ArchiveOldUnitsJob#perform`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/archive_old_units_job.rb#L6-L24) – old-unit archiving raises an exception | Configured administrator/error recipient | Operational path; no user preference | +| D2L grade transfer result | [`D2lPostGradesJob#perform`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/d2l_post_grades_job.rb#L9-L37) – D2L grade transfer completes | User who initiated the transfer | Direct workflow result; no notification preference check | +| D2L grade transfer failure | [`D2lPostGradesJob#perform`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/d2l_post_grades_job.rb#L9-L37) – D2L grade transfer fails | User who initiated the transfer | Direct workflow result; no notification preference check | +| Communication email to student | [`ExecuteCommunicationSetJob#execute_email_student_action`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/execute_communication_set_job.rb#L116-L159) – an active communication rule matches a student and executes its student-email action | Student matched by the communication rule | No v2 preference check; requires rule match, configured action, recipient email and sender email | +| Communication email to staff | [`ExecuteCommunicationSetJob#execute_email_staff_action`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/execute_communication_set_job.rb#L163-L198) – a staff-email action executes | Tutors and/or convenors selected by the rule | No v2 preference check; requires configured recipient groups, available recipient addresses and sender email | +| Communication action log | [`ExecuteCommunicationSetJob#send_action_log_to_convenors`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/execute_communication_set_job.rb#L264-L311) – a communication execution produces its action log | Convenors | Requires `send_log_to_convenors?`, convenor addresses and sender email; no v2 preference check | +| Tutor note | [`NotifyTutorNotesJob#perform`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/app/sidekiq/notify_tutor_notes_job.rb#L4-L10) – tutor-note notification job runs | Specific recipient supplied to the job | No preference check in this job path | +| PDF-generation error mail | [`submission:generate_pdfs`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/lib/tasks/generate_pdfs.rake#L86-L149) – portfolio/PDF generation raises an exception | Configured administrator/error recipient | Operational path; no user preference | +| Portfolio ready | [`submission:generate_pdfs`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/lib/tasks/generate_pdfs.rake#L86-L149) – portfolio generation succeeds | Project student | `receive_portfolio_notifications` | +| Portfolio failed | [`submission:generate_pdfs`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/lib/tasks/generate_pdfs.rake#L86-L149) – portfolio generation fails | Project student | `receive_portfolio_notifications` | +| Task PDF failed – maintenance | [`notify_failed_submission`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/lib/tasks/maintenance.rake#L42-L68) – maintenance PDF processing identifies a failed submission | Project student | `receive_task_notifications` | +| Maintenance error mail | [`notify_failed_submission`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/lib/tasks/maintenance.rake#L42-L68) – maintenance processing raises an error while handling the failure | Configured administrator/error recipient | Operational path; no user preference | +| Overseer assessment failed | [`notify_failed_overseer_assessments!`](https://github.com/ontrack-features-t2-2026/doubtfire-api/blob/09a61714425f12e1412da5b7a34f31f7ea5612dd/lib/tasks/overseer_notifications.rake#L2-L18) – unnotified Overseer assessment failures are grouped for delivery | Affected project student | Requires queued failure records and a nonblank student email; no explicit user preference check in this method | + +## Mailers already present + +The API currently contains the following relevant mailers: + +- `CommunicationsMailer` – sends configurable communication emails and + communication action logs. +- `D2lResultMailer` – reports D2L grade-transfer results. +- `ErrorLogMailer` – sends operational/error reports. +- `NotificationsMailer` – sends single event notifications and weekly student + and staff summaries. +- `PortfolioEvidenceMailer` – handles task PDF failure, task feedback ready, + Overseer assessment failure, portfolio ready and portfolio failed emails. +- `TutorNoteMailer` – sends tutor-note notifications to a supplied recipient. + +`ConvenorContactMailer#request_project_membership` and +`PortfolioEvidenceMailer#task_pdf_ready_message` also exist, but no active +`.deliver`/`.deliver_now` send site was found for either during this audit. +They are therefore not counted among the 21 current email send sites. + +## Existing web communication subsystem + +The web application already contains a unit communications editor under: + +`src/app/units/states/edit/directives/unit-communications-editor/` + +This is an existing communication system rather than a placeholder for future +notification work. + +A convenor can configure communication rules with conditions and actions. +Available actions include: + +- Send email to student +- Send email to staff +- Add a task comment +- Change target grade + +Student emails support a configurable subject and body. Staff emails also +support configurable subject/body content and can target tutors, convenors, or +both. + +Communication rules can filter students using existing conditions including +task status, target grade, login status, special consideration, tutorial, +tutorial stream and campus. + +The subsystem also supports scheduled communication sets. A schedule can run +once or recur daily, weekly or monthly. It supports a start week/day/time, +timezone, recurrence interval, repeat count and optional end date. + +When a communication set executes, matched students can receive the configured +actions. The execution logic also prevents a student matched by an earlier rule +in the same set from being processed again by a later rule. + +## Duplicate-email risks + +The existing communications subsystem is the largest duplication risk for v2. +OnTrack can already send configurable email to students and staff, including +scheduled and recurring communication across a unit. A new event should not +reimplement this behaviour without first deciding whether the event belongs in +the existing communication-rule system. + +Portfolio events are another clear overlap. OnTrack already emails a student +when portfolio generation succeeds and when it fails. A v2 portfolio event that +also sends email could therefore double-mail the same student. + +Task and feedback events must also be checked against +`PortfolioEvidenceMailer`, weekly summaries and the communication-rule system. +Existing task-PDF failures and feedback-ready messages already reach students. + +Tutor-note and task-comment work also require care. Tutor notes already have a +direct email path, while the communication editor can create task comments. +New notification hooks around these actions should establish whether the +existing email is being replaced, supplemented, or intentionally left alone. + +`NotificationService` introduces another duplication boundary. Events routed +through it can generate a notification email after the relevant notification +preference check. An event must not also retain an independent legacy email +unless two messages are explicitly intended. + +The current target branch also includes the `task_status_changed` event. This +event calls `NotificationService.notify`, so it reuses the single notification +email delivery path listed above. It does not introduce a separate direct +`.deliver`, `.deliver_now`, or `.deliver_later` call and therefore does not +increase the direct send-site count. + +## Recipient and preference observations + +Existing emails do not use one common preference mechanism. + +Task-PDF failure paths use `receive_task_notifications`. The batch feedback-ready +email, weekly student summary and weekly staff summary use +`receive_feedback_notifications`. Portfolio-ready and portfolio-failed emails +use `receive_portfolio_notifications`. + +`NotificationService` only maps `task`, `feedback`, and `portfolio` to existing +preference fields. The `extension` and `general` types have no preference +mapping, so `NotificationService.deliver_to?` currently allows them by default. + +Communication-rule emails do not use the v2 preference mapping. They are +controlled by rule matching, action configuration, available recipient +addresses and an available sender address. The action-log email also requires +the rule's `send_log_to_convenors?` setting and at least one convenor email. + +Administrator error emails and D2L result emails have no user notification +preference check. Error emails depend on the operational error-email +configuration, while D2L result emails are sent directly to the user who +initiated the transfer. + +This distinction must be preserved when an existing email is migrated or +connected to a v2 event. Adding a second preference check without understanding +the legacy path could suppress a required operational email. Keeping both an +independent legacy send and a v2 send could instead cause duplicate user-facing +email. + +## Conclusion + +OnTrack already has substantial email functionality in both repositories. + +In particular: + +1. Students are already emailed when their portfolio is ready or generation + fails. +2. Students already receive task, feedback, summary and Overseer-related + emails in existing flows. +3. Staff already receive weekly summaries and can be targeted through the unit + communications system. +4. Convenors can already configure and schedule email to a unit without any + new v2 notification feature. +5. The communication subsystem supports both student and staff email and + recurring schedules. +6. New v2 events must be checked against these send paths before another email + channel is added. + +The safest rule for subsequent event tickets is therefore: before adding an +email delivery path, check this audit and the existing communication subsystem +to determine whether OnTrack already sends an equivalent message. diff --git a/docs/notifications/push-opt-in-permission-flow.md b/docs/notifications/push-opt-in-permission-flow.md new file mode 100644 index 0000000000..d2c7399380 --- /dev/null +++ b/docs/notifications/push-opt-in-permission-flow.md @@ -0,0 +1,270 @@ +# Push opt-in and permission flow + +MN-D02. This document records the user-visible flow for enabling and disabling +Web Push on one device, the browser and server state behind it, and the recovery +path for each known failure. + +This is a documentation-only ticket. It does not change permission, subscription, +delivery, or sign-out behaviour. + +Push is device-specific. Turning it on stores this browser's subscription for +the signed-in user; it does not opt every browser or phone into push. Existing +notification category preferences still apply. For example, turning task +notifications off prevents task notifications on every channel, even when this +device remains subscribed to push. + +For server configuration, payload shape, and delivery diagnostics, read +[`push-setup.md`](./push-setup.md). For secure-context rules, service-worker +cleanup, and phone tunnel setup, read +[`testing-push-locally.md`](./testing-push-locally.md). + +## Happy path + +The browser permission prompt must follow a user action. OnTrack therefore asks +for permission only after the user clicks the push setting; it never prompts on +sign-in or page load. + +```text +Sign in + -> open Profile + -> wait for the service worker to start + -> click "Turn on push notifications on this device" + -> grant the browser's notification permission + -> browser creates a PushSubscription using the server's VAPID public key + -> web app POSTs endpoint, p256dh, and auth to /api/push_subscriptions + -> api stores or updates the subscription for the signed-in user + -> another user action raises a notification event + -> api sends the push through the stored endpoint + -> service worker displays the notification +``` + +In more detail: + +1. The user signs in and opens **Profile**. The push control is part of the + edit-profile form, below the notification category settings. +2. The Angular service worker registers about six seconds after application + bootstrap. Until it is ready, the button is disabled and the page says: + **Still starting up. This becomes available a few seconds after the page + loads.** +3. `PushNotificationService.blocker()` checks that the browser exposes the + Notifications and Push APIs, permission has not already been denied, VAPID + is configured, and `SwPush` is enabled. +4. The user clicks **Turn on push notifications on this device**. The button is + disabled while the request is running so a second click cannot create a + competing request. +5. `SwPush.requestSubscription` asks the browser for permission and supplies + the VAPID public key published by the api. +6. After permission is granted, the browser returns a subscription. The web app + posts its `endpoint`, `p256dh`, and `auth` values to the authenticated + `POST /api/push_subscriptions` endpoint. +7. The api validates the endpoint as a recognised HTTPS push service and stores + it for the current user. Posting the same endpoint again updates its keys + rather than creating a duplicate. If the endpoint previously belonged to + another account on the same browser, ownership moves to the current user. +8. The page shows **This device will receive push notifications**, the button + changes to **Turn off push notifications on this device**, and a toast says + **Push notifications turned on**. +9. When an enabled notification event is raised, + `NotificationService.notify` creates the in-app notification and calls the + push delivery service. The service sends the Angular notification payload to + every stored subscription for the recipient. `ngsw-worker.js` displays it. + +Permission and subscription are different state. Browser permission allows the +site to show notifications. The `PushSubscription` gives the api a destination +and encryption keys. A user needs both, plus an active service worker and an +enabled notification category, before an event can appear as a push. + +## What the user sees + +| State | Push control | User-visible result | Recovery | +| ------------------------------------ | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| Service worker is starting | Disabled | **Still starting up. This becomes available a few seconds after the page loads.** | Wait at least six seconds. If it persists, follow the service-worker checks below. | +| Ready, permission is `default` | Enabled | **Turn on push notifications on this device** | Click the button and choose **Allow** in the browser prompt. | +| Subscribing | Disabled | The existing button remains visible while the request runs. | Wait for the success or error toast; do not reload during the prompt. | +| Subscribed | Enabled | **Turn off push notifications on this device**, plus **This device will receive push notifications.** | No action is needed. A real event is still required to prove delivery. | +| Permission denied | Disabled | **You have blocked notifications for this site**, followed by browser-specific steps for Chrome, Edge, or Firefox, or generic steps for another browser. | Allow notifications in site settings and reload. A site cannot override a denial or prompt again by itself. | +| Push API unsupported | Disabled | **This browser does not support push notifications.** | Use a browser and device that expose Web Push in a secure context, and check that platform's installation requirements. | +| VAPID not configured | Disabled | **Push notifications are not set up on this server.** | An operator must configure the api. The user cannot correct this in the browser. | +| Request fails after a denial | Disabled on the next state check | Toast: **Notifications are blocked in your browser**. | Change the site's notification permission to Allow, then reload. | +| Other subscribe or unsubscribe error | Depends on the browser subscription that remains | Toast: **Could not change push notifications**. | Check the api response and logs, then retry. Do not treat the toast as evidence that local and server state match. | + +The browser-specific denial instructions come from +`PERMISSION_DENIED_INSTRUCTIONS` in the web push service. Opera and unrecognised +browsers deliberately use the generic instructions rather than Chrome's steps. + +## Failure and recovery paths + +### Permission was denied + +`Notification.permission === 'denied'` is a hard blocker. OnTrack disables the +button because browsers do not let a site reverse that decision or show the +permission prompt again. The page explains how to reopen the site's permission +settings for Chrome, Edge, and Firefox. After changing the permission, reload so +the blocker and the local subscription are read again. + +If the user denies the first prompt, `requestSubscription` rejects. The immediate +feedback is the **Notifications are blocked in your browser** toast; after the +state refresh or reload, the disabled control and recovery steps explain what to +do next. + +### The browser or context is unsupported + +OnTrack reports **This browser does not support push notifications** when either +`Notification` or `PushManager` is absent. This can mean the browser genuinely +lacks Web Push, but it can also mean the page is outside a secure context. + +`http://localhost` is treated as trustworthy for local development. A phone +opened at a laptop's plain HTTP LAN address is not. Use HTTPS for a real device +and check: + +```js +window.isSecureContext && "serviceWorker" in navigator; +``` + +The result must be `true`. The phone and tunnel procedure is in +[`testing-push-locally.md`](./testing-push-locally.md). + +### The service worker is missing or stuck + +`SwPush.isEnabled` is false during the normal six-second registration delay and +when the current build did not generate or register `ngsw-worker.js`. Both cases +show the same temporary starting-up message. + +If the button stays disabled: + +1. Request `/ngsw-worker.js` and confirm it returns `200`, not `404`. +2. Confirm the browser shows an activated service worker for the current origin. +3. Clear stale registrations and caches using + `doubtfire-web/docs/service-worker.md` or the browser-specific steps in + [`testing-push-locally.md`](./testing-push-locally.md). +4. Reload and wait for registration before reopening the push setting. + +### The subscription expired or was invalidated + +When a push service answers `404` or `410`, the api treats the registration as +dead and deletes its `push_subscriptions` row. Temporary failures stay inside +the asynchronous delivery path: they are raised to Sidekiq for retry and never +propagate to the original request, in-app record, or email hand-off. + +There is currently no message back to the open browser when this cleanup +happens. Its local `SwPush.subscription` can therefore still make Profile say +**This device will receive push notifications** even though the api no longer +has a destination. The symptom is a subscribed-looking device that receives no +push and has no row on the api. + +To recover today, click **Turn off push notifications on this device** and then +turn it on again. The web app still removes the local subscription when the api +delete returns `404`, so a fresh opt-in can create and store a new endpoint. +Automatic re-registration when a browser rotates its subscription belongs to +MN-C05; this flow does not claim it already happens. + +### The operating system muted the browser + +Browser permission can be `granted` while the operating system blocks the +browser's notifications, Focus or Do Not Disturb suppresses them, or the alert +style shows no banner. The web app cannot see those operating-system settings. +It continues to show **This device will receive push notifications**, and the +api can successfully send, while nothing appears on screen. + +Use a local notification to separate an operating-system problem from an +OnTrack delivery problem: + +```js +const registration = await navigator.serviceWorker.ready; +await registration.showNotification("OnTrack", { + body: "Local notification test; no api or push service involved", +}); +``` + +If this does not appear, allow notifications for the browser in the operating +system, choose a visible alert style, and turn off Focus or Do Not Disturb. If it +does appear, continue with the subscription, recipient, VAPID, and api-log checks +in [`push-setup.md`](./push-setup.md). + +### A delivery service or api call failed + +Temporary push errors are logged and the server keeps the subscription. One +failing browser never blocks delivery to the recipient's other devices and +never blocks the in-app notification or email. + +A failed opt-in API request produces **Could not change push notifications**. +Because the browser may already have created its local subscription before the +POST fails, verify both sides rather than relying on the toast: + +```js +await navigator.serviceWorker.ready.then((registration) => + registration.pushManager.getSubscription(), +); +``` + +```sh +docker exec doubtfire-api bundle exec rails runner \ + 'puts PushSubscription.all.map { |s| "#{s.user.username} #{s.endpoint[0, 60]}" }' +``` + +If the local subscription exists but the api row does not, turn push off and on +after correcting the api error. + +## Revoking push on this device + +Clicking **Turn off push notifications on this device** performs two operations +in this order: + +1. `DELETE /api/push_subscriptions?endpoint=...` removes the signed-in user's + server row while the auth token and endpoint still exist. +2. `SwPush.unsubscribe()` removes the browser subscription. + +Server deletion goes first because local unsubscribe discards the endpoint the +api needs to find the row. If the delete fails, the web app still unsubscribes +locally. The leftover server row is harmless and is removed when a later send +receives `404` or `410`. On success, the toast says **Push notifications turned +off** and the control returns to its opt-in state. + +Revoking the site's permission directly in browser settings is different from +using the OnTrack button. The page can detect that permission is now denied, but +the current code does not proactively delete the api row in response to a +permission change. Browser cleanup varies; any dead row is removed after the +push service reports it as expired or invalid. Use the OnTrack control when +possible so both sides are cleaned up deliberately. + +## Sign-out ordering + +Sign-out also removes push because a browser subscription survives an ordinary +web session. Leaving it behind on a shared device could send the previous +user's notifications after another person signs in. + +`AuthenticationService.signOut` therefore: + +1. calls `PushNotificationService.unsubscribeQuietly()` while the current + user's auth token still exists; +2. deletes the api row before unsubscribing in the browser; +3. continues deleting the server session and local auth token whether push + cleanup succeeds or fails; and +4. completes sign-out even when the service worker is disabled or the browser + refuses to unsubscribe. + +The quiet variant is deliberate: push cleanup protects a shared device, but an +outage must never trap someone in a signed-in session. When there is no active +service worker, sign-out cannot read a local endpoint and returns immediately; +any server row it could not address is left for delivery-time cleanup. + +## Source map + +| Responsibility | Source | +| ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| Push control, status text, busy state, and toasts | `doubtfire-web/src/app/common/edit-profile-form/edit-profile-form.component.ts` and `.html` | +| Blocker order, browser guidance, subscribe, and unsubscribe | `doubtfire-web/src/app/api/services/push-notification.service.ts` | +| Service-worker registration delay | `doubtfire-web/src/app/doubtfire-angular.module.ts` | +| Sign-out cleanup ordering | `doubtfire-web/src/app/api/services/authentication.service.ts` | +| Authenticated store, update, ownership move, and delete endpoints | `doubtfire-api/app/api/push_subscriptions_api.rb` | +| Endpoint validation and uniqueness | `doubtfire-api/app/models/push_subscription.rb` | +| Push fan-out, payload, expired-row cleanup, and error isolation | `doubtfire-api/app/services/push_notification_service.rb` | + +## Verification boundary + +The service and API tests cover the state decisions and request ordering, but +they cannot prove that a browser or operating system displayed a notification. +A complete manual verification must record the browser and OS versions, grant +permission through the Profile control, confirm the api row, trigger a real +event for the subscribed user, observe the notification, and test its click +destination. Use the local testing guide for that end-to-end procedure. diff --git a/docs/notifications/push-setup.md b/docs/notifications/push-setup.md new file mode 100644 index 0000000000..c1cf4ae3c0 --- /dev/null +++ b/docs/notifications/push-setup.md @@ -0,0 +1,173 @@ +> Production startup now requires a valid matching VAPID pair. Both keys may be +> unset only in development/test. See [delivery operations](delivery-operations.md) +> for secret injection, key rotation and required client resubscription. + +# Web push setup + +How the push channel works, how to turn it on, and how to check it is working. + +## What push needs + +Three things, and push is a no-op until all three are true: + +1. **VAPID keys on the api.** Without them `PushNotificationService.configured?` + is false and `deliver` returns immediately. The app behaves exactly as it did + before push existed. +2. **A row in `push_subscriptions`.** A browser has to register itself first. + MN-F01 added the table and the API. +3. **A service worker in the browser.** MN-F03 turns it on for development. + Without it, the browser has nothing to receive a push with. Setup, + caching side effects and how to clear a stuck worker are in the web repo: + `doubtfire-web/docs/service-worker.md`. + +Miss any one and nothing arrives, with no error anywhere. Check them in order. + +## The keys + +VAPID is how a push service knows the push came from us and not from anyone else +who happens to know a browser's endpoint URL. It is one key pair for the whole +server, not one per user. + +Generate a pair: + + docker exec doubtfire-api bundle exec ruby -e \ + "require 'web_push'; k = WebPush.generate_key; puts k.public_key; puts k.private_key" + +Then set three environment variables on the api: + +| Variable | What it is | +|---|---| +| `DOUBTFIRE_VAPID_PUBLIC_KEY` | Public half. The browser needs this to subscribe. | +| `DOUBTFIRE_VAPID_PRIVATE_KEY` | **Secret.** Signs every push. Never commit a real one. | +| `DOUBTFIRE_VAPID_SUBJECT` | A `mailto:` or URL the push service can contact. Optional; falls back to the institution host. | + +`development/docker-compose.yml` in the deploy repo already carries a throwaway +pair so the local stack works out of the box, on the same footing as +`DF_SECRET_KEY_BASE`. That pair is development only. **A production deployment +sets its own through real secrets, and if the private key ever leaks, generate a +new pair — every existing subscription becomes useless and users have to +re-subscribe.** + +Changing the keys does not migrate anything. The `push_subscriptions` rows stay, +but pushes signed with the new key are rejected for browsers that subscribed +under the old one. The delivery service deliberately retains generic failures, +including 403 responses, because they can also be transient configuration +errors. When rotating a VAPID pair, explicitly delete all existing +`push_subscriptions` rows and tell users to enable push again. + +## How a notification becomes a push + +`NotificationService.notify` queues `PushNotificationDeliveryJob` with only the +notification id. A Sidekiq worker reloads the notification and calls +`PushNotificationService.deliver`, so **every event that queues an email also +queues a push, with no per-event work**. Provider network I/O never blocks the +request or runs under the notification hand-off lock. Push jobs use the +dedicated `notifications` queue; every environment that enables Web Push must +run a worker for that queue. + +`deliver` loops over `notification.user.push_subscriptions` and sends this +payload to each: + +```json +{ + "notification": { + "title": "OnTrack", + "body": "Andrew Cain commented on 1.1P in COS10001.", + "data": { "notification_id": 12, "link": "/projects/2/dashboard/1.1P" } + } +} +``` + +The top level `notification` key matters. Angular's own `ngsw-worker.js` looks +for exactly that and displays the notification itself. **Change the shape and +somebody has to write a service worker by hand.** + +`data.link` is what MN-C03 reads to decide where to send the user on click. +Delivery uses normal urgency and a one-hour TTL so a short disconnect can +recover without a push service surfacing workflow alerts days or weeks late. + +## Failure handling + +- **404 or 410** means the browser threw the registration away. The row is + deleted, because nothing will ever reach it again. +- **Anything else** (429 rate limit, the push service being down, a rejected + payload) is logged and retained. Delivery continues to the user's remaining + browsers, then the aggregate failure is raised so Sidekiq can retry. The + subscription is kept because deleting on a temporary outage would silently + unsubscribe people the first time a push service had a bad day. +- The delivery job is configured for three Sidekiq retries. A missing + notification row is also retryable because a fast worker may run before an + enclosing producer transaction commits. +- Nothing propagates to the request caller. A push failure must never block the + in-app notification or the email hand-off. + +## Checking it works + +**Are the keys loaded?** + + docker exec doubtfire-api bundle exec rails runner \ + 'puts PushNotificationService.configured?' + +`false` means the api container was started before the variables were added. +`restart` does not pick up new environment variables. Recreate it: + + docker compose -f docker-compose.yml -f docker-compose.local-paths.yml up -d doubtfire-api + +**Is a browser registered?** + + docker exec doubtfire-api bundle exec rails runner \ + 'puts PushSubscription.all.map { |s| "#{s.user.username} #{s.endpoint[0, 60]}" }' + +Empty means nothing has subscribed yet. That is the usual reason a push does not +arrive, and it looks identical to push being broken. + +**Subscribe this browser by hand for diagnostics.** The normal path is the +in-app opt-in control. To isolate that UI from the API and service worker, paste +this into the dev tools console on a page where you are signed in. + +```js +const VAPID = '<the public key>' +const b64 = s => Uint8Array.from(atob(s.replace(/-/g,'+').replace(/_/g,'/')), c => c.charCodeAt(0)) + +const reg = await navigator.serviceWorker.ready +const sub = await reg.pushManager.subscribe({ + userVisibleOnly: true, + applicationServerKey: b64(VAPID) +}) +const j = sub.toJSON() + +await fetch('/api/push_subscriptions', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'Username': localStorage.getItem('username'), + 'Auth-Token': localStorage.getItem('authToken') + }, + body: JSON.stringify({ endpoint: j.endpoint, p256dh: j.keys.p256dh, auth: j.keys.auth }) +}) +``` + +Then raise a notification (post a task comment) and a desktop notification +should appear. + +**Nothing appeared?** Check in this order, because each step is invisible when it +fails: + +1. Browser notification permission. `Notification.permission` must be `granted`. + macOS also has to allow notifications from the browser, in System Settings. +2. `PushNotificationService.configured?` is true. +3. A `push_subscriptions` row exists for the user the notification went to. It is + easy to subscribe as one account and then trigger a notification for another. +4. The Sidekiq worker consumes the `notifications` queue. The normal development + stack starts it with `-q mailers -q notifications`; it deliberately does not + consume the unrelated `default` queue. +5. `docker logs doubtfire-sidekiq | grep -i "push"`. Delivery failures happen in + the worker, so this is the process that reports them. + +## Why the gem is pinned + +`Gemfile` pins `web-push` to exactly `3.0.1`. This release still depends on +`jwt ~> 2.0`, so it remains compatible with the authentication dependencies in +this release, while replacing the separate `hkdf` gem with `OpenSSL::KDF`. +`web-push` 3.0.2 is the release that moves to JWT 3; upgrading beyond 3.0.1 +therefore needs authentication and OAuth regression testing. diff --git a/docs/notifications/pwa-offline.md b/docs/notifications/pwa-offline.md new file mode 100644 index 0000000000..bcaf16252f --- /dev/null +++ b/docs/notifications/pwa-offline.md @@ -0,0 +1,264 @@ +# PWA offline behaviour + +This document records what users experience when OnTrack loses network +connectivity and explains the relevant Angular service-worker configuration. + +No caching configuration or application behaviour was changed as part of this +documentation-only investigation. + +## Configuration under test + +Testing used the generated static development build from the +`doubtfire-web/feature/notifications` branch. + +Before the final offline tests, the environment confirmed that: + +- `navigator.serviceWorker.controller` referenced `ngsw-worker.js`. +- `/ngsw/state` reported `Driver state: NORMAL ((nominal))`. +- `/index.html` existed in the versioned `app` asset cache. +- **Bypass for network** was disabled. +- **Update on reload** was disabled. +- Ordinary reloads were used rather than forced or hard refreshes. + +The generated static build was used for the final test so the service worker +could install and cache the built application files consistently. + +## What is cached + +### Application shell + +The `app` asset group uses `installMode: "prefetch"` and includes: + +- `/index.html` +- the compiled JavaScript bundles +- the compiled CSS bundles +- the favicon +- the web application manifest + +These resources are downloaded when the service-worker application version is +installed. + +The `assets` asset group uses `installMode: "lazy"`. Matching images, fonts, +and other assets are cached after they are requested rather than all being +downloaded during installation. + +### Navigation URLs + +The service-worker configuration contains these navigation rules: + +```json +[ + "/**", + "!/**/*.*", + "!/**/*__*", + "!/**/*__*/**", + "!/JPlag/**", + "!/JPlag", + "!/sidekiq/**", + "!/sidekiq", + "!/beta/**", + "!/beta", + "!/legacy", + "!/legacy/**" +] +``` + +A URL must match a positive rule and must not match any negative rule to be +treated as an Angular navigation request. + +The `!/**/*.*` rule is intended to exclude file URLs whose final path segment +contains a file extension. However, it also excludes valid OnTrack task routes +when the task abbreviation contains a period. + +For example: + +- `/` is treated as a navigation request. +- `/projects/2/dashboard` is treated as a navigation request. +- `/projects/28/dashboard/A15` is treated as a navigation request. +- `/projects/2/dashboard/2.2P` is not treated as a navigation request because + its final path segment contains a period. + +No `navigationRequestStrategy` override is configured. Angular therefore uses +its default `performance` navigation strategy for matching navigation +requests. This strategy serves the configured `/index.html`, which is normally +available from the application cache. + +An excluded URL is not redirected to the cached index file. When the network +is unavailable, the excluded request cannot be completed and may produce the +browser's native error page. + +### API data + +The `api` data group uses the following policy: + +```json +{ + "name": "api", + "urls": ["/api"], + "cacheConfig": { + "maxSize": 0, + "maxAge": "0u", + "strategy": "freshness" + } +} +``` + +`freshness` is a network-first strategy. The zero maximum size and zero maximum +age indicate that matching responses are not intended to provide a reusable +offline API cache. + +Users should therefore not rely on grades, task state, submission details, +comments, prerequisites, or other API-backed information being available +after connectivity is lost. + +This configuration should not be described as proof that an API response can +never be written to or returned from a service-worker cache under any +condition. Its practical effect and intent are that API data should not be +relied upon for meaningful offline reuse. + +A successful response status such as `200` or `304` does not, by itself, +identify where the response came from. The Network panel's **Size**, +**Transferred**, and **Initiator** information must be inspected before +identifying a response as coming from the service worker, memory cache, disk +cache, or network. + +## Observed offline behaviour + +### Included navigation route + +An ordinary offline reload of `/` loaded the cached Angular application shell +rather than Chrome's native `HTTP ERROR 504` page. + +Static resources such as the favicon and application icon were returned by the +service worker. Authentication, API, analytics, and other uncached requests +failed while the browser was offline. + +The application shell could therefore load, but API-backed application state +was unavailable or incomplete. This confirms that caching the shell does not +provide a complete offline mode. + +### Dotted task route + +An ordinary offline reload of: + +```text +/projects/2/dashboard/2.2P +``` + +produced Chrome's native `HTTP ERROR 504` page. + +Before this test: + +- `ngsw-worker.js` controlled the page. +- The service-worker driver state was normal. +- `/index.html` existed in the application cache. +- The reload was an ordinary reload rather than a forced refresh. + +The route was excluded from Angular navigation handling because its final +segment, `2.2P`, contains a period and matches the `!/**/*.*` negative rule. + +The service worker therefore did not use cached `/index.html` as the +navigation fallback. With the network unavailable, the route request failed +and Chrome displayed its native error page. + +This is a route-specific navigation limitation. It is not evidence that the +application shell was missing from the service-worker cache. + +### Losing connectivity mid-session + +When connectivity was removed without reloading, the already-loaded +application shell and information held in memory remained visible. + +Requests for new API-backed information failed. This included secondary task +requests such as prerequisite, submission-detail, and comment requests when +the information had not already been loaded. + +One observed failure produced this toast: + +> Failed to fetch prerequisites for task definition: TypeError: Cannot read +> properties of null (reading 'error') + +This is a technical error rather than a clear explanation that the application +has lost network connectivity. + +Some previously requested resources may still return successful statuses while +offline. Their source must be confirmed using the Network panel rather than +being inferred from the status code alone. + +## Summary + +OnTrack caches its Angular application shell, but offline reload behaviour is +route-dependent. + +Routes that satisfy the configured navigation rules can receive cached +`/index.html`. Valid task routes whose final path segment contains a period are +excluded by `!/**/*.*` and can produce a browser-level `504` when reloaded +offline. + +Even when the shell loads, API-backed information cannot be relied upon +offline. Losing connectivity can therefore leave the application shell visible +while new data requests fail and technical errors are displayed. + +The current behaviour can result in three different user experiences: + +1. An included navigation route loads the cached application shell, but + API-backed information is missing or fails. +2. A dotted task route produces Chrome's native `504` page because it is + excluded from navigation fallback. +3. Losing connectivity during an existing session leaves the shell visible + but can produce failed requests and technical error messages. + +## Recommendation + +A separate `doubtfire-web` ticket should investigate narrowing or replacing +the `!/**/*.*` navigation exclusion so valid task abbreviations containing +periods can use the Angular navigation fallback without treating genuine +static-file requests as application routes. + +A separate ticket should also consider: + +- displaying an offline or disconnected status; +- replacing technical request errors with user-friendly messages; +- safely handling absent network responses; +- providing a retry path after connectivity returns; and +- identifying which information, if any, should be available offline. + +These changes are outside the scope of this documentation-only ticket. + +## How to verify manually + +1. Build and serve the generated Angular output. +2. Load OnTrack while online. +3. Wait until the service worker is ready. +4. Confirm that + `navigator.serviceWorker.controller?.scriptURL` + ends in `ngsw-worker.js`. +5. Confirm that `/ngsw/state` reports + `Driver state: NORMAL ((nominal))`. +6. Confirm that `/index.html` exists in the versioned `app` asset cache. +7. Ensure **Bypass for network** is disabled. +8. Ensure **Update on reload** is disabled. +9. Load `/` while online. +10. Set Network throttling to **Offline**. +11. Use ordinary Command+R and confirm that the cached Angular shell loads. +12. Do not use Shift+Command+R or **Empty cache and hard reload**. +13. Return Network throttling to **No throttling**. +14. Load `/projects/2/dashboard/2.2P`. +15. Set Network throttling back to **Offline**. +16. Use ordinary Command+R. +17. Confirm that the dotted task route produces Chrome's native `504` page. +18. Return online and load a task. +19. Remove connectivity without reloading. +20. Navigate to information that has not already been requested. +21. Record the failed API requests and any user-facing error. +22. Use the Network panel's **Size**, **Transferred**, and **Initiator** + information before attributing successful responses to a particular cache. + +## Relevant configuration + +The behaviour described here is controlled primarily by: + +- `doubtfire-web/ngsw-config.json` +- the generated `ngsw.json` service-worker manifest +- Angular's service-worker navigation request handling +- the application's handling of failed API requests diff --git a/docs/notifications/testing-push-locally.md b/docs/notifications/testing-push-locally.md new file mode 100644 index 0000000000..77e265211a --- /dev/null +++ b/docs/notifications/testing-push-locally.md @@ -0,0 +1,507 @@ +# Testing push notifications locally + +MN-D03. How to get a push notification to arrive on your own machine, and then on +a real phone. + +This does not cover generating VAPID keys, registering a browser by hand, or what +the payload looks like. That is all in +[`push-setup.md`](./push-setup.md), in this same folder. Read that first. This document is only about the two +things it does not answer: why push works on `localhost` with no HTTPS, and why it +stops working the moment you point a phone at your laptop. + +Every section is marked **Verified** or **Not tested**. Read those marks. The +tunnel half of this guide has not been walked end to end by anyone yet, so treat +section 3 as a route rather than a proven path and correct it as you go. + +--- + +## 0. Push does nothing without MN-F03 + +**Verified** — read from web#4, `push/enable-service-worker-in-dev`, merged into +`feature/notifications` in `doubtfire-web` on 8 Aug 2026. + +A service worker is the thing that receives a push. Before MN-F03, `angular.json` +set `"serviceWorker": "ngsw-config.json"` only on the `production` build +configuration, so a development build never generated `ngsw-worker.js` at all. +The browser had nothing to receive a push with, and the failure was silent. + +MN-F03 changed two files: + +```diff + "development": { + "optimization": false, + "extractLicenses": false, +- "sourceMap": true ++ "sourceMap": true, ++ "serviceWorker": "ngsw-config.json" + }, +``` + +```diff + ServiceWorkerModule.register('ngsw-worker.js', { +- enabled: environment.production, ++ enabled: environment.production || environment.enableServiceWorker, + registrationStrategy: () => interval(6000).pipe(take(1)), + }), +``` + +plus `enableServiceWorker: true` in `src/environments/environment.ts`. + +**If you are on a branch that does not have MN-F03, nothing in this guide will +work and you will get no error telling you why.** Check before you start: + + curl -s -o /dev/null -w "%{http_code}\n" http://localhost:4200/ngsw-worker.js + +200 means you have it. 404 means you do not. Full detail, including the six second +registration delay and the `$NODE_ENV` trap, is in +`docs/service-worker.md` in the `doubtfire-web` repo, on `feature/notifications`. + +--- + +## 1. `http://localhost` is a secure context, so localhost needs no HTTPS + +**Verified** — behaviour recorded in `doubtfire-web/docs/service-worker.md` on +2026-08-02, where a push sent from the api arrived as a desktop notification on +`http://localhost:4200`. I did not re-run it for this guide. + +Service workers and the Push API are restricted to secure contexts. People read +that as "I need HTTPS" and go and generate a self-signed certificate, or set up +mkcert, or ask why the dev stack does not do TLS. **None of that is necessary.** + +The rule is not "must be HTTPS". It is "must be a *potentially trustworthy +origin*", and loopback addresses are on that list by definition. So all of these +are secure contexts: + +- `http://localhost:4200` +- `http://localhost` on any port +- `http://127.0.0.1:4200` +- `http://[::1]:4200` + +Which means the ordinary dev stack, `ng serve` on port 4200 over plain HTTP, is +already good enough to register a service worker, subscribe to push, and receive +a push. Do this part first. It is the fast loop, and if push does not work here it +will not work anywhere else either. + +Order of work: + +1. `curl` `/ngsw-worker.js` and get a 200 (section 0). +2. Follow `push-setup.md` to confirm the keys are loaded and subscribe the browser. +3. Trigger an event, get a notification on your desktop. + +Only once that works should you go anywhere near a tunnel. + +--- + +## 2. The trap: a phone on your wifi is not a secure context + +**Verified on a physical phone.** MN-Q03 reproduced this on an iPhone 16 +running iOS 26.6 on 13 Aug 2026. The phone reached the api over the LAN +address, but the push opt-in remained disabled because the page was not a +secure context. + +`angular.json` sets the dev server to `"host": "0.0.0.0"`, so `ng serve` listens +on every interface, not just loopback. Your laptop's LAN address works. You can +type `http://192.168.1.42:4200` into a phone on the same wifi and the OnTrack app +loads, logs in, and behaves completely normally. + +**And push will not work, and nothing will tell you why.** + +`192.168.1.42` is not a loopback address. It is a plain HTTP origin like any +other, so it is not a secure context, so `navigator.serviceWorker` is not even +defined on that page. Practically: + +- No service worker registers. +- `SwPush.isEnabled` is `false`, so MN-C01's opt-in button reports "not + supported" rather than an error. +- `Notification.requestPermission()` may still work, which makes it look like + permissions are the problem when they are not. +- Nothing appears in the console. There is no exception, no warning, no failed + request. The feature is just absent. + +This is the evening-costing one. The symptom is "it works on my laptop but not on +my phone", and the instinct is to go and debug the subscription code, the VAPID +key, the api logs, notification permissions on the phone. All of that is fine. The +page is simply not a secure context. + +Quick check, in the phone's browser console or as a bookmarklet: + +```js +console.log(window.isSecureContext, 'serviceWorker' in navigator); +``` + +`false false` on the LAN address, `true true` through a tunnel. If you only take +one thing from this document, take that line. + +**Testing on a real phone therefore needs real HTTPS, which means a tunnel.** + +### iOS is a second trap on top of the first + +**Not tested.** Documented Safari behaviour, included because it will come up. + +Safari on iOS only supports Web Push for web apps that have been added to the +Home Screen. Opening the tunnel URL in Safari and expecting a push will fail even +over HTTPS. The user has to Share → Add to Home Screen and open it from there. +Android Chrome has no such restriction. If you are picking a phone to test with, +pick Android. + +--- + +## 3. Tunnel setup with cloudflared + +**Not tested.** `cloudflared` is not installed on this machine and I have not run +any of this. The steps below are written from the tool's documented behaviour and +from configuration I did verify in our repos (each config change is marked +separately). Treat the sequence as a first draft that needs someone to walk it. + +I picked `cloudflared` over `ngrok` because a quick tunnel needs no account, no +signup and no authtoken. `ngrok` now requires an account before it will forward +anything. One tool, done properly, rather than two done badly. + +### Why one tunnel is enough + +**Verified** — read from `doubtfire-web/src/app/config/constants/hostUrl.ts` and +`doubtfire-web/proxy.conf.json`. + +The obvious worry is that you need two tunnels, one for the web app on 4200 and +one for the api on 3000, and that the phone would load an HTTPS page that then +tries to call `http://localhost:3000` and gets blocked as mixed content. + +That does not happen, because of two things already in the repo: + +```ts +// src/app/config/constants/hostUrl.ts +const HOST_URL: string = `${window.location.protocol}//${window.location.hostname}${window.location.port ? ':' + window.location.port : ''}`; +``` + +The app derives its api base URL from wherever the page was loaded from. It is not +hardcoded. And `package.json` runs `ng serve ... --proxy-config proxy.conf.json`, +which forwards `/api` to the api container: + +```json +{ "/api": { "target": "http://localhost:3000", "secure": false } } +``` + +So the browser only ever talks to one origin. Tunnel port 4200 and the api comes +along with it. **Do not tunnel port 3000 as well.** It will not help and it gives +you a second hostname to get wrong. + +### Step 1 — install cloudflared + + brew install cloudflared + +### Step 2 — have the stack running on localhost first + +Section 1. If push does not work on `http://localhost:4200`, a tunnel will not fix +it, it will just add a second thing that can be broken. + +### Step 3 — start the tunnel + + cloudflared tunnel --url http://localhost:4200 + +It prints a hostname that looks like: + + https://random-words-here.trycloudflare.com + +That hostname is new every time you restart the tunnel. Which matters, because +both config changes below name it, so **you will be editing config every time you +restart the tunnel.** Leave it running. + +### Step 4 — let the Angular dev server answer to that hostname + +**Verified** that this option exists and is spelled this way — read from +`node_modules/@angular/build/src/builders/dev-server/schema.json` at version +22.0.4. Not verified that the tunnel then works. + +Vite, which is what `@angular/build:dev-server` runs on, rejects requests whose +`Host` header is not in its allowlist. Without this you get a Vite "Blocked +request" page through the tunnel instead of the app. + +In `angular.json`, under `projects.doubtfire.architect.serve.options`: + +```json +"serve": { + "builder": "@angular/build:dev-server", + "options": { + "buildTarget": "doubtfire:build", + "port": 4200, + "host": "0.0.0.0", + "allowedHosts": ["random-words-here.trycloudflare.com"] + }, +``` + +Changing `angular.json` does not update the already-running dev server. Restart +the web service before opening the tunnel hostname. Run this from +`doubtfire-deploy/development`: + + docker compose -f docker-compose.yml -f docker-compose.local-paths.yml \ + restart doubtfire-web + +If the frontend is running directly with `npm start`, stop it and start it again +instead. + +The schema also accepts `"allowedHosts": true` to allow everything. Its own +description calls that "not recommended and a security risk", which is fair, since +your dev server is on the public internet for as long as the tunnel is up. Use it +if you are restarting the tunnel constantly and are sick of editing this file, but +do not commit it. + +**Do not commit any of this.** It is a hostname that will not exist tomorrow. + +### Step 5 — let the api answer to that hostname + +**Verified** — I reproduced the failure directly against the running api +container: + +``` +$ curl -s -H "Host: probe-test.trycloudflare.com" http://localhost:3000/api/settings +Blocked hosts: probe-test.trycloudflare.com +To allow requests to these hosts, make sure they are valid hostnames (containing +only numbers, letters, dashes and dots), then add the following to your +environment configuration: +config.hosts << "probe-test.trycloudflare.com" +``` + +The same request with the default `Host` returns 200. + +This is Rails 8 Host Authorization, not CORS. In development Rails allows loopback +addresses, any raw IP, and anything ending in `.localhost` or `.test`. A +`trycloudflare.com` hostname is none of those, so the api rejects it before any +of our code runs. + +The fix that needs no code change, in `doubtfire-deploy/development/docker-compose.yml` +under `doubtfire-api.environment`: + +```yaml + RAILS_DEVELOPMENT_HOSTS: 'random-words-here.trycloudflare.com' +``` + +**Verified** that Rails reads this variable and splits it on commas — railties +8.0.2, `lib/rails/application/configuration.rb`. Comma separate if you need more +than one. + +Then recreate the container. `restart` does not pick up new environment +variables, same as with the VAPID keys: + + docker compose -f docker-compose.yml -f docker-compose.local-paths.yml up -d doubtfire-api + +Confirm it took: + + curl -s -o /dev/null -w "%{http_code}\n" \ + -H "Host: random-words-here.trycloudflare.com" \ + http://localhost:3000/api/settings + +### About CORS, which is not your problem + +**Verified** — read from `doubtfire-api/config/application.rb:287`. + + config.middleware.insert_before Warden::Manager, Rack::Cors do + allow do + origins '*' + resource '*', headers: :any, methods: %i(get post put delete options) + end + end + +Origins is already `*`. **There is no CORS change to make for a tunnel.** And +because of the dev server proxy in step 3, the api calls are same-origin anyway, +so CORS is not even in play. If you are looking at a CORS error you have found a +different bug. The api-side change you actually need is `RAILS_DEVELOPMENT_HOSTS` +above. + +### An alternative to step 5 that I have not tried + +**Not tested.** Setting `"changeOrigin": true` on the `/api` entry in +`proxy.conf.json` should make the proxy rewrite the `Host` header to +`localhost:3000` before forwarding, so Rails never sees the tunnel hostname and +`RAILS_DEVELOPMENT_HOSTS` becomes unnecessary. That would survive tunnel restarts, +which is the appeal. + +I did not test it, and I did not confirm what the proxy's default actually is. +`RAILS_DEVELOPMENT_HOSTS` is the one I verified fails and can be made to pass, so +that is what step 5 says. If you try `changeOrigin`, note that in the Docker stack +`doubtfire-deploy/development/docker-compose.local-paths.yml` mounts +`proxy.conf.docker.json` over the repo's `proxy.conf.json` read-only, so you have +to edit the deploy repo's copy, not the web repo's. + +### Step 6 — open it on the phone + +Open the printed `https://...trycloudflare.com` URL on the phone. Check the secure +context first, before anything else: + +```js +window.isSecureContext && 'serviceWorker' in navigator +``` + +Before subscribing on the phone, record the existing rows: + + docker exec doubtfire-api bundle exec rails runner \ + 'puts PushSubscription.order(:id).map { |s| "#{s.id} #{s.user.username} #{s.endpoint[0, 80]}" }' + +Then sign in, wait out the six second service worker registration delay, and use +MN-C01's opt-in button. Run the same command again. The new or changed row is +the phone's subscription. + +Do not identify the device only from the endpoint host. It narrows the browser +but it does not name the device, and Chromium does not imply one service: +measured on 27 Aug 2026, Chrome 152 subscribed through `fcm.googleapis.com` and +Edge 151 on macOS through `wns2-bl2p.notify.windows.com`. Firefox uses Mozilla +Push and Safari/iOS uses Apple Web Push. Comparing the rows before and after +subscribing is the reliable check. Then trigger an event as another user and +watch. + +### Things that will probably go wrong + +**Not tested.** Written from what the configuration implies, not from experience. + +- **Vite's HMR websocket may not survive the tunnel.** The page still loads, live + reload just stops. Not worth fixing for a push test, reload by hand. +- **The tunnel hostname changes on every restart**, and both step 4 and step 5 + name it. If push worked yesterday and does not today, check that first. +- **A quick tunnel is public.** Anyone with the URL reaches your dev stack with + its throwaway VAPID keys and seeded database. Stop it when you are done. +- **You will have two subscriptions for your user**, one from the desktop test and + one from the phone. Both get pushed. That is correct behaviour, not a bug. + +--- + +## 4. Clearing a stuck service worker + +**Not tested** in Firefox. The Chrome console snippet is taken from +`doubtfire-web/docs/service-worker.md`, which is where the fuller writeup of the +service worker's caching side effects lives. + +A service worker caches the whole app bundle and keeps serving it. The symptom is +that you change code, reload, and still see the old code. A hard reload does not +help, because the worker still intercepts the request. + +### Both browsers, from the console + +Works in Chrome and Firefox. Faster than the UI and the one to reach for at 1am: + +```js +(await navigator.serviceWorker.getRegistrations()).forEach((r) => r.unregister()); +const keys = await caches.keys(); +await Promise.all(keys.map((k) => caches.delete(k))); +location.reload(); +``` + +### Chrome, through dev tools + +1. F12 → **Application** → **Service Workers**. +2. **Unregister** next to `ngsw-worker.js`. +3. **Application** → **Storage** → **Clear site data**. +4. Reload. + +While you are actively working on the app, **Application → Service Workers → +Bypass for network** stops the worker serving cached responses without +unregistering it. That is usually what you want during normal development. It is a +per-devtools-session setting and it resets when you close dev tools. + +`chrome://serviceworker-internals` lists every registration in the profile and +will unregister them, which is the one to use when a worker is stuck on an origin +you no longer have open. + +### Firefox, through dev tools + +1. F12 → **Application** → **Service Workers**. +2. **Unregister**. +3. **Storage** → right click the origin → **Delete All**. +4. Reload. + +`about:debugging#/runtime/this-firefox` is the equivalent of Chrome's +`serviceworker-internals` and has **Unregister** buttons per worker. + +Firefox private windows do not run service workers at all, so push cannot work +there. Do not use one to test. + +--- + +## 5. Resetting notification permission + +**Not tested.** Written from the current browser UIs. Someone should walk these +and correct them. + +Permission is per origin, and once denied the browser will not ask again. The +opt-in button will report "blocked" forever and no amount of clicking will +prompt. You have to reset it by hand. Everyone denies it by accident once. + +Check where you stand, in the console: + +```js +Notification.permission // "default" | "granted" | "denied" +``` + +`default` means you will be prompted. `denied` means you will not. + +### Chrome + +Fastest: click the icon at the left of the address bar (the tune or lock icon), +find **Notifications**, set it back to **Ask (default)**. Reload. + +Or `chrome://settings/content/notifications`, find the origin under **Not allowed +to send notifications**, and remove it. Whole-origin nuke, which also clears the +service worker and everything else, is **Clear site data** in the same panel. + +Two things that are not the same as the browser permission and get confused with +it: + +- **macOS System Settings → Notifications → Google Chrome.** If Chrome itself is + not allowed to post notifications, the browser permission can be `granted` and + the push can arrive and you still see nothing. `push-setup.md` calls this out + too. Check it once, then stop thinking about it. +- **Focus / Do Not Disturb.** Same result, notifications delivered silently to + Notification Centre. + +### Firefox + +Click the padlock in the address bar → **Clear cookies and site data**, or expand +**Connection secure** → **More information** → **Permissions**, find **Receive +Notifications**, and untick **Use Default** then set it back. + +Or `about:preferences#privacy` → **Permissions** → **Notifications** → +**Settings**, find the origin, **Remove Website**. Reload. + +Check `about:preferences#privacy` → **Notifications** → **Settings** for **Block +new requests asking to allow notifications** as well. If that is ticked, nothing +will ever prompt and the state will read `denied` on every site. + +### Android Chrome + +Site permissions are under the padlock → **Permissions** → **Notifications**. But +also check Android **Settings → Apps → Chrome → Notifications**, because if +Chrome as an app is blocked at the OS level then no site inside it can post +anything, and the in-page permission will still say `granted`. + +--- + +## Verification status, all in one place + +| Section | Status | +|---|---| +| MN-F03 is required, and what it changed | **Verified.** Read from the web#4 diff, merged 8 Aug 2026. | +| `http://localhost` is a secure context | **Verified.** Recorded working in `doubtfire-web/docs/service-worker.md`, 2026-08-02. Not re-run here. | +| A LAN address is not a secure context | **Verified on a physical phone.** iPhone 16 / iOS 26.6, 13 Aug 2026, MN-Q03. The phone reached the api over the LAN address but the opt-in button was disabled with "This browser does not support push notifications". | +| iOS needs Add to Home Screen | **Not tested.** Documented Safari behaviour. | +| One tunnel is enough, because of `hostUrl.ts` + the dev server proxy | Config **verified** by reading it. The conclusion is an inference, **not tested**. | +| `cloudflared` install and tunnel steps | **Not tested.** `cloudflared` is not installed on this machine. | +| `allowedHosts` is a real dev server option | **Verified** against `@angular/build` 22.0.4's schema. Effect through a tunnel **not tested**. | +| Rails blocks a tunnel hostname | **Verified.** Reproduced with `curl -H "Host: ..."` against the running api. | +| `RAILS_DEVELOPMENT_HOSTS` is the variable Rails reads | **Verified** in railties 8.0.2 source. Not tested end to end through a tunnel. | +| No CORS change is needed | **Verified.** `origins '*'` at `config/application.rb:287`. | +| `changeOrigin` as an alternative | **Not tested.** Offered as a lead, not a step. | +| Clearing a service worker | Console snippet from `service-worker.md`. Chrome and Firefox UI paths **not tested**. | +| Resetting notification permission | **Not tested**, all browsers. | + +Section 2 has now been walked. MN-Q03 hit exactly the failure this guide +predicts, on an iPhone 16 running iOS 26.6 on 13 Aug 2026. The phone reached the +api fine over the LAN address and the opt-in button was still disabled, which is +the browser refusing to expose the push API outside a secure context. So the +rule holds on real hardware and not just on paper. + +Section 3, the tunnel, is still unwalked. Nobody has yet run `cloudflared` end +to end and got a push onto a phone. That is what MN-Q02 and the retest of MN-Q03 +are for, and a screenshot of a notification arriving on a real lock screen is +the deliverable that closes them. + +If you are the first person through section 3, correct this document as you go +rather than working around it. Every step in there was reasoned from config +rather than executed, and the marks in the table above say which is which. diff --git a/lib/helpers/database_populator.rb b/lib/helpers/database_populator.rb index b415a30ec2..589e52d3a7 100644 --- a/lib/helpers/database_populator.rb +++ b/lib/helpers/database_populator.rb @@ -175,8 +175,12 @@ def generate_overseer_images tag: 'bash:latest' ) - echo_line "---> Pulling overseer image #{overseer_image.tag}" - overseer_image.pull_from_docker + if ENV['SKIP_OVERSEER_IMAGE_PULL_ON_POPULATE'] == 'true' + echo_line "---> Skipping overseer image pull for #{overseer_image.tag}" + else + echo_line "---> Pulling overseer image #{overseer_image.tag}" + overseer_image.pull_from_docker + end end # @@ -211,10 +215,9 @@ def generate_users(filter = nil) if AuthenticationHelpers.aaf_auth? user = User.create!(profile) else - user = User.create!(profile.merge({ - password: 'password', - password_confirmation: 'password' - })) + user = User.new(profile) + user.password = 'password' + user.save! end @user_cache[user_key] = user @@ -651,7 +654,7 @@ def generate_tasks_for_unit(unit, unit_details) if (File.exist? csv_to_import) && (File.exist? zip_to_import) echo "----> CSV file found, importing tasks from #{csv_to_import} \n" - result = unit.import_tasks_from_csv(File.open(csv_to_import)) + result = unit.import_tasks_from_csv(File.open(csv_to_import), notify: false) unless result[:errors].empty? raise("----> Task import from CSV failed with the following errors: #{result[:errors]} \n") end diff --git a/lib/shell/sidekiq_entry_point.sh b/lib/shell/sidekiq_entry_point.sh index 7a44f7bd50..d32595094e 100755 --- a/lib/shell/sidekiq_entry_point.sh +++ b/lib/shell/sidekiq_entry_point.sh @@ -1,5 +1,7 @@ #!/bin/bash +set -euo pipefail + # Start the run once job. echo "Sidekiq docker container has been started" @@ -9,14 +11,16 @@ newaliases # Setup msmptrc if [ -f "/shared-files/msmtprc" ]; then echo "Copying msmtprc file from shared-files" - cp -f /shared-files/msmtprc /etc; + install -o root -g root -m 0600 /shared-files/msmtprc /etc/msmtprc else echo "msmtprc file not found in shared-files, using default configuration" fi -# Ensure mail settings are accessible only by root -chown root:root /etc/msmtprc -chmod 600 /etc/msmtprc +# Ensure existing mail settings are accessible only by root. +if [ -f /etc/msmtprc ]; then + chown root:root /etc/msmtprc + chmod 0600 /etc/msmtprc +fi -# Run sidekiq -bundle exec sidekiq +# Make Sidekiq PID 1 so Docker stop signals reach it directly. +exec bundle exec sidekiq diff --git a/lib/shell/write_cron_environment.sh b/lib/shell/write_cron_environment.sh new file mode 100755 index 0000000000..4bc0b194af --- /dev/null +++ b/lib/shell/write_cron_environment.sh @@ -0,0 +1,52 @@ +#!/bin/bash + +set -euo pipefail + +# Cron starts jobs with a deliberately small environment. Persist only the +# application and Ruby runtime settings that those jobs can need, rather than +# copying every variable inherited by the container. Values are shell-escaped +# because this file is sourced through BASH_ENV by .ci-setup/crontab. +is_cron_environment_variable() { + case "$1" in + BUNDLE_* | DATABASE_URL | DF_* | D2L_* | DISK_SPACE_ENDPOINT_ENABLED | \ + DOCKER_CERT_PATH | DOCKER_HOST | DOCKER_PROXY_URL | DOCKER_REGISTRY_URL | \ + DOCKER_TLS_VERIFY | DOCKER_TOKEN | DOCKER_USER | DOUBTFIRE_* | GEM_HOME | \ + GEM_PATH | GOTENBERG_* | HTTP_PROXY | HTTPS_PROXY | LANG | LATEX_* | \ + LC_* | LTI_* | MODERATION_SCORE_FACTOR | NO_PROXY | OVERSEER_* | \ + RABBITMQ_* | RACK_ENV | RAILS_* | RUBYLIB | RUBYOPT | SENTRY_* | \ + SSL_CERT_DIR | SSL_CERT_FILE | TCA_* | TII_* | TMPDIR | TZ | \ + http_proxy | https_proxy | no_proxy) + return 0 + ;; + *) + return 1 + ;; + esac +} + +destination="${1:-/container.env}" +temporary_file='' + +cleanup() { + if [[ -n "${temporary_file}" ]]; then + rm -f -- "${temporary_file}" + fi +} + +trap cleanup EXIT +trap 'exit 1' HUP INT TERM + +umask 077 +temporary_file="$(mktemp "${destination}.tmp.XXXXXX")" + +while IFS= read -r variable_name; do + if is_cron_environment_variable "${variable_name}"; then + printf 'export %s=%q\n' \ + "${variable_name}" "${!variable_name}" >> "${temporary_file}" + fi +done < <(compgen -e | LC_ALL=C sort) + +chmod 0600 "${temporary_file}" +mv -f -- "${temporary_file}" "${destination}" +temporary_file='' +trap - EXIT HUP INT TERM diff --git a/lib/tasks/notification_delivery.rake b/lib/tasks/notification_delivery.rake new file mode 100644 index 0000000000..7a7b5140d5 --- /dev/null +++ b/lib/tasks/notification_delivery.rake @@ -0,0 +1,20 @@ +# frozen_string_literal: true + +namespace :notifications do + desc 'Count persisted email delivery outcomes without exposing recipient data' + task delivery_counts: :environment do + puts Notification.group(:email_delivery_state).count.to_json + end + + desc 'Requeue one investigated failed email: NOTIFICATION_ID=123' + task retry_email: :environment do + notification = Notification.find(Integer(ENV.fetch('NOTIFICATION_ID'), 10)) + notification.with_lock do + abort 'Only failed or queue_failed emails can be retried' unless %w[failed queue_failed].include?(notification.email_delivery_state) + + notification.update!(email_delivery_state: 'pending', email_delivery_error_class: nil) + end + NotificationService.queue_email(notification) + puts({ notification_id: notification.id, email_delivery_state: notification.reload.email_delivery_state }.to_json) + end +end diff --git a/lib/tasks/send_digests.rake b/lib/tasks/send_digests.rake new file mode 100644 index 0000000000..4c5fc42397 --- /dev/null +++ b/lib/tasks/send_digests.rake @@ -0,0 +1,19 @@ +namespace :mailer do + # The student digest, one mail per student across every unit they are in. + # + # The scheduled runs are in config/schedule.yml and go through + # SendDigestEmailsJob. This is the same sweep, run now and in this process, + # for an operator who wants one cadence sent by hand. Running it does not + # double up on a scheduled run, because the job claims each student for the + # period before mailing them. + # + # CADENCE picks which students are in scope: it must match their chosen + # digest_frequency. 'off' is rejected, because a run with that cadence would + # mail the students who asked for nothing. + # + # This is not mailer:send_status_emails. That one is the older per-unit mail + # and it still sends what it always sent. + task send_digests: :environment do + SendDigestEmailsJob.new.perform(ENV.fetch('CADENCE', 'weekly')) + end +end diff --git a/lib/tasks/send_status_emails.rake b/lib/tasks/send_status_emails.rake index 98ce225b57..53cd2442b6 100644 --- a/lib/tasks/send_status_emails.rake +++ b/lib/tasks/send_status_emails.rake @@ -1,9 +1,31 @@ namespace :mailer do + # The older per-unit summary, one mail per student per unit. + # + # Nothing schedules this any more. It ran from .ci-setup/crontab every Monday + # at 7am and that line is gone: config/schedule.yml now runs the student + # digest instead, which is one mail covering all of a student's units. Both on + # a schedule would mean a student gets the digest and a mail per unit in the + # same morning. Whether the digest supersedes this outright is a call for the + # team, so this is left working and reachable by hand rather than deleted. + # + # CADENCE picks which students are in scope: it must match their chosen + # digest_frequency. 'off' is rejected, because User#wants_digest_on? only + # compares the two strings, so that run would mail exactly the students who + # asked for no summary at all. + # + # There is no guard on this one. It sends with deliver_now and records + # nothing, so running it twice sends two copies. The digest has a guard; + # see DigestDeliveryGuard. task send_status_emails: :environment do summary_stats = {} + cadence = ENV.fetch('CADENCE', 'weekly') + raise ArgumentError, "CADENCE must be one of #{SendDigestEmailsJob::CADENCES.join(', ')}" unless SendDigestEmailsJob::CADENCES.include?(cadence) + + summary_stats[:cadence] = cadence + summary_stats[:week_end] = Time.zone.now - summary_stats[:week_start] = summary_stats[:week_end] - 7.days + summary_stats[:week_start] = summary_stats[:week_end] - { 'daily' => 1.day, 'weekly' => 7.days, 'monthly' => 1.month }.fetch(cadence, 7.days) summary_stats[:weeks_comments] = TaskComment.where("created_at >= :start AND created_at < :end", start: summary_stats[:week_start], end: summary_stats[:week_end]).count summary_stats[:weeks_engagements] = TaskEngagement.where("engagement_time >= :start AND engagement_time < :end", start: summary_stats[:week_start], end: summary_stats[:week_end]).count diff --git a/lib/vapid_configuration.rb b/lib/vapid_configuration.rb new file mode 100644 index 0000000000..f1807a8d43 --- /dev/null +++ b/lib/vapid_configuration.rb @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +require 'base64' +require 'web-push' + +module VapidConfiguration + def self.validate!(production:, environment: ENV) + public_key = environment['DOUBTFIRE_VAPID_PUBLIC_KEY'].to_s + private_key = environment['DOUBTFIRE_VAPID_PRIVATE_KEY'].to_s + return if !production && public_key.empty? && private_key.empty? + + if public_key.empty? || private_key.empty? + raise ArgumentError, 'Configure DOUBTFIRE_VAPID_PUBLIC_KEY and DOUBTFIRE_VAPID_PRIVATE_KEY before enabling production push' + end + + begin + decoded_public = Base64.urlsafe_decode64(public_key) + decoded_private = Base64.urlsafe_decode64(private_key) + raise ArgumentError unless decoded_public.bytesize == 65 && decoded_public.getbyte(0) == 4 && decoded_private.bytesize == 32 + + key = WebPush::VapidKey.from_keys(public_key, private_key) + raise ArgumentError unless key.curve.check_key + rescue StandardError + # Never include a key, OpenSSL diagnostic, or exception cause in boot logs. + raise ArgumentError, 'Invalid VAPID key pair; configure a matching P-256 public/private pair', cause: nil + end + end +end diff --git a/test/api/additional_notification_emails_api_test.rb b/test/api/additional_notification_emails_api_test.rb new file mode 100644 index 0000000000..c63c57082d --- /dev/null +++ b/test/api/additional_notification_emails_api_test.rb @@ -0,0 +1,79 @@ +require 'test_helper' + +class AdditionalNotificationEmailsApiTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + + def app + Rails.application + end + + setup do + @user = FactoryBot.create(:user, email: 'primary@example.edu') + @other_user = FactoryBot.create(:user) + add_auth_header_for(user: @user) + end + + def test_current_user_can_request_read_and_remove_an_address + put_json "/api/users/#{@user.id}/additional_notification_email", { + email: 'secondary@example.org' + } + + assert_equal 200, last_response.status + assert_equal 'pending', last_response_body['status'] + assert_equal 'secondary@example.org', last_response_body['email'] + + get "/api/users/#{@user.id}/additional_notification_email" + assert_equal 200, last_response.status + assert_equal 'private, no-store', last_response.headers['Cache-Control'] + + delete "/api/users/#{@user.id}/additional_notification_email" + assert_equal 204, last_response.status + assert_nil @user.reload.additional_notification_email + end + + def test_a_user_cannot_manage_another_users_destination + put_json "/api/users/#{@other_user.id}/additional_notification_email", { + email: 'secondary@example.org' + } + + assert_equal 403, last_response.status + assert_nil @other_user.reload.additional_notification_email + end + + def test_public_verification_consumes_the_token_once + record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + token = record.verification_token + clear_auth_header + + post_json '/api/additional_notification_emails/verify', { token: token } + assert_equal 201, last_response.status + assert_equal 'verified', last_response_body['status'] + + post_json '/api/additional_notification_emails/verify', { token: token } + assert_equal 409, last_response.status + end + + def test_verification_token_is_filtered_from_server_logs + assert_includes Rails.application.config.filter_parameters, :token + end + + def test_rate_limit_returns_429_without_queueing_another_message + put_json "/api/users/#{@user.id}/additional_notification_email", { + email: 'secondary@example.org' + } + 2.times do + post_json "/api/users/#{@user.id}/additional_notification_email/resend", {} + end + queued = AdditionalNotificationEmailVerificationJob.jobs.count + + post_json "/api/users/#{@user.id}/additional_notification_email/resend", {} + + assert_equal 429, last_response.status + assert_equal queued, AdditionalNotificationEmailVerificationJob.jobs.count + end +end diff --git a/test/api/notifications_api_test.rb b/test/api/notifications_api_test.rb new file mode 100644 index 0000000000..002c5f4dfd --- /dev/null +++ b/test/api/notifications_api_test.rb @@ -0,0 +1,249 @@ +require 'test_helper' + +# EN-T02: the notifications API endpoints in app/api/notifications_api.rb. +# +# Six routes: +# GET /api/notifications (list, optional unread_only) +# GET /api/notifications/unread_count +# PUT /api/notifications/:id/read +# PUT /api/notifications/read_all +# DELETE /api/notifications (through_id confirmation boundary) +# DELETE /api/notifications/:id +# +# Every route scopes through current_user.notifications, so one user can never +# touch another's notifications. That is the case worth proving. +# +# Run this file on its own, not the whole suite: the test database is the +# development database (DF_TEST_DB_DATABASE == doubtfire-dev), so a full run +# holds locks and rewrites seeded data. See item 11 in +# doubtfire-deploy/RUNNING-LOCALLY.md. +class NotificationsApiTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + + def app + Rails.application + end + + setup do + @user = FactoryBot.create(:user, :student) + @other = FactoryBot.create(:user, :student) + end + + # GET /api/notifications ---------------------------------------------------- + + def test_list_returns_the_users_notifications_newest_first + older = FactoryBot.create(:notification, user: @user, created_at: 2.days.ago) + newer = FactoryBot.create(:notification, user: @user, created_at: 1.hour.ago) + FactoryBot.create(:notification, user: @other) # must not appear + + add_auth_header_for(user: @user) + get '/api/notifications' + + assert_equal 200, last_response.status + + json = JSON.parse(last_response.body) + ids = json.map { |n| n['id'] } + + assert_equal 2, json.length, 'only the current user\'s notifications' + assert_equal [newer.id, older.id], ids, 'recent_first order' + end + + def test_list_includes_the_event_the_client_routes_on + FactoryBot.create(:notification, user: @user, event: 'task_comment_created') + + add_auth_header_for(user: @user) + get '/api/notifications' + + assert_equal 200, last_response.status + assert_equal 'task_comment_created', JSON.parse(last_response.body).first['event'] + end + + def test_list_unread_only_filters_out_read_notifications + unread = FactoryBot.create(:notification, user: @user) + FactoryBot.create(:notification, :read, user: @user) + + add_auth_header_for(user: @user) + get '/api/notifications', unread_only: true + + assert_equal 200, last_response.status + + json = JSON.parse(last_response.body) + + assert_equal 1, json.length + assert_equal unread.id, json.first['id'] + end + + # GET /api/notifications/unread_count --------------------------------------- + + def test_unread_count_counts_only_the_users_unread + FactoryBot.create_list(:notification, 2, user: @user) # unread + FactoryBot.create(:notification, :read, user: @user) # read, excluded + FactoryBot.create(:notification, user: @other) # other user, excluded + + add_auth_header_for(user: @user) + get '/api/notifications/unread_count' + + assert_equal 200, last_response.status + assert_equal 2, JSON.parse(last_response.body)['count'] + end + + # PUT /api/notifications/:id/read ------------------------------------------- + + def test_marking_a_notification_as_read + notification = FactoryBot.create(:notification, user: @user) + + add_auth_header_for(user: @user) + put "/api/notifications/#{notification.id}/read" + + assert_equal 200, last_response.status + assert_not_nil JSON.parse(last_response.body)['read_at'] + assert_not_nil notification.reload.read_at + end + + # PUT /api/notifications/read_all ------------------------------------------- + + def test_marking_all_as_read_clears_only_the_users_unread + FactoryBot.create_list(:notification, 3, user: @user) + others = FactoryBot.create(:notification, user: @other) + + add_auth_header_for(user: @user) + put '/api/notifications/read_all' + + assert_equal 200, last_response.status + assert JSON.parse(last_response.body)['success'] + assert_equal 0, @user.notifications.unread.count + assert_nil others.reload.read_at, 'another user\'s notifications are untouched' + end + + # DELETE /api/notifications ------------------------------------------------- + + def test_deleting_all_uses_the_confirmed_boundary_and_current_user_scope + first = FactoryBot.create(:notification, user: @user) + # Created inside the boundary, so only the current_user scope can save it. + other_users = FactoryBot.create(:notification, user: @other) + through = FactoryBot.create(:notification, user: @user) + unseen_newer = FactoryBot.create(:notification, user: @user) + assert_operator other_users.id, :<, through.id + + add_auth_header_for(user: @user) + delete '/api/notifications', through_id: through.id + + assert_equal 200, last_response.status + assert_equal 2, JSON.parse(last_response.body)['deleted_count'] + assert_not Notification.exists?(first.id) + assert_not Notification.exists?(through.id) + assert Notification.exists?(unseen_newer.id), 'a notification after confirmation is retained' + assert Notification.exists?(other_users.id), 'another user is untouched' + end + + def test_deleting_all_rejects_a_missing_negative_or_malformed_boundary + notification = FactoryBot.create(:notification, user: @user) + + add_auth_header_for(user: @user) + + [{}, { through_id: -1 }, { through_id: 'all' }].each do |params| + assert_no_difference 'Notification.count' do + delete '/api/notifications', params + end + + assert_equal 400, last_response.status, "expected 400 for #{params.inspect}" + end + assert Notification.exists?(notification.id) + end + + def test_deleting_all_requires_a_positive_confirmation_boundary + notification = FactoryBot.create(:notification, user: @user) + + add_auth_header_for(user: @user) + + assert_no_difference 'Notification.count' do + delete '/api/notifications', through_id: 0 + end + + assert_equal 400, last_response.status + assert Notification.exists?(notification.id) + end + + # DELETE /api/notifications/:id --------------------------------------------- + + def test_deleting_a_notification + notification = FactoryBot.create(:notification, user: @user) + + add_auth_header_for(user: @user) + + assert_difference 'Notification.count', -1 do + delete "/api/notifications/#{notification.id}" + end + + assert_equal 200, last_response.status + end + + # Cross-user isolation ------------------------------------------------------ + + def test_a_user_cannot_mark_another_users_notification_as_read + theirs = FactoryBot.create(:notification, user: @other) + + add_auth_header_for(user: @user) + put "/api/notifications/#{theirs.id}/read" + + assert_equal 404, last_response.status + assert_nil theirs.reload.read_at, 'it must stay unread' + end + + def test_a_user_cannot_delete_another_users_notification + theirs = FactoryBot.create(:notification, user: @other) + + add_auth_header_for(user: @user) + + assert_no_difference 'Notification.count' do + delete "/api/notifications/#{theirs.id}" + end + + assert_equal 404, last_response.status + end + + # NB-02: the bell clears itself --------------------------------------------- + + # The production path, end to end and through HTTP. Nothing here touches the + # bell: the student loads the task's comments the way the web app does, and + # the unread count drops on its own. This is the before-and-after the card + # asks for, in a form a reviewer can rerun. + def test_loading_a_tasks_comments_drops_the_students_unread_count + project = FactoryBot.create(:project) + task_definition = project.unit.task_definitions.first + task = project.task_for_task_definition(task_definition) + student = project.student + tutor = project.tutor_for(task_definition) + + task.add_text_comment(tutor, 'Have a look at question three.') + + add_auth_header_for(user: student) + get '/api/notifications/unread_count' + + assert_equal 200, last_response.status + assert_equal 1, JSON.parse(last_response.body)['count'], 'before the comments load' + + add_auth_header_for(user: student) + get "/api/projects/#{project.id}/task_def_id/#{task_definition.id}/comments" + + assert_equal 200, last_response.status + + add_auth_header_for(user: student) + get '/api/notifications/unread_count' + + assert_equal 200, last_response.status + assert_equal 0, JSON.parse(last_response.body)['count'], 'after the comments load' + end + + # Authentication ------------------------------------------------------------ + + def test_an_unauthenticated_request_is_rejected + clear_auth_header + + get '/api/notifications' + + assert_equal 419, last_response.status + end +end diff --git a/test/api/push_subscriptions_api_test.rb b/test/api/push_subscriptions_api_test.rb new file mode 100644 index 0000000000..3323fb97e7 --- /dev/null +++ b/test/api/push_subscriptions_api_test.rb @@ -0,0 +1,161 @@ +require 'test_helper' + +# MN-F01: storing a browser's push registration. +class PushSubscriptionsApiTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + + def app + Rails.application + end + + setup do + @user = FactoryBot.create(:user, :student) + @other = FactoryBot.create(:user, :student) + end + + def test_a_user_can_register_a_browser + params = FactoryBot.attributes_for(:push_subscription) + + add_auth_header_for(user: @user) + + assert_difference 'PushSubscription.count', 1 do + post '/api/push_subscriptions', params + end + + assert_equal 201, last_response.status + + subscription = PushSubscription.last + + assert_equal @user, subscription.user + assert_equal params[:endpoint], subscription.endpoint + end + + def test_the_response_does_not_leak_the_browser_keys + params = FactoryBot.attributes_for(:push_subscription) + + add_auth_header_for(user: @user) + post '/api/push_subscriptions', params + + json = JSON.parse(last_response.body) + + assert_equal params[:endpoint], json['endpoint'] + assert_not json.key?('p256dh'), 'the browser public key must not be sent back' + assert_not json.key?('auth'), 'the browser auth secret must not be sent back' + end + + # Same params both times, so the endpoint has to be built once and reused. The + # factory sequences it, so calling the factory twice would be a different + # browser and this would test nothing. + def test_registering_the_same_browser_twice_updates_instead_of_duplicating + params = FactoryBot.attributes_for(:push_subscription) + + add_auth_header_for(user: @user) + post '/api/push_subscriptions', params + + # Generate an actual public point; arbitrary 65-byte strings are not keys. + key = OpenSSL::PKey::EC.generate('prime256v1') + rotated_key = Base64.urlsafe_encode64(key.public_key.to_octet_string(:uncompressed)) + + assert_no_difference 'PushSubscription.count' do + post '/api/push_subscriptions', params.merge(p256dh: rotated_key) + end + + assert_equal rotated_key, PushSubscription.last.p256dh + end + + # Shared machine. The endpoint belongs to the browser, so the registration has + # to move to whoever signed in last rather than blowing up on the unique index. + def test_registering_a_browser_another_user_had_moves_it_across + subscription = FactoryBot.create(:push_subscription, user: @other) + + add_auth_header_for(user: @user) + + assert_no_difference 'PushSubscription.count' do + post '/api/push_subscriptions', subscription.slice(:endpoint, :p256dh, :auth) + end + + assert_equal @user, subscription.reload.user + assert_empty @other.push_subscriptions.reload + end + + def test_a_user_only_sees_their_own_registrations + mine = FactoryBot.create(:push_subscription, user: @user) + FactoryBot.create(:push_subscription, user: @other) + + add_auth_header_for(user: @user) + get '/api/push_subscriptions' + + assert_equal 200, last_response.status + + json = JSON.parse(last_response.body) + + assert_equal 1, json.length + assert_equal mine.endpoint, json.first['endpoint'] + end + + def test_a_user_can_remove_their_own_registration + subscription = FactoryBot.create(:push_subscription, user: @user) + + add_auth_header_for(user: @user) + + assert_difference 'PushSubscription.count', -1 do + delete '/api/push_subscriptions', endpoint: subscription.endpoint + end + + assert_equal 200, last_response.status + end + + def test_a_user_cannot_remove_someone_elses_registration + subscription = FactoryBot.create(:push_subscription, user: @other) + + add_auth_header_for(user: @user) + + assert_no_difference 'PushSubscription.count' do + delete '/api/push_subscriptions', endpoint: subscription.endpoint + end + + assert_equal 404, last_response.status + end + + def test_an_unauthenticated_request_is_rejected + clear_auth_header + + assert_no_difference 'PushSubscription.count' do + post '/api/push_subscriptions', FactoryBot.attributes_for(:push_subscription) + end + + assert_equal 419, last_response.status + end + + def test_a_registration_missing_the_browser_keys_is_rejected + add_auth_header_for(user: @user) + + assert_no_difference 'PushSubscription.count' do + post '/api/push_subscriptions', FactoryBot.attributes_for(:push_subscription).except(:p256dh, :auth) + end + + assert_equal 400, last_response.status + end + + # Firefox and Safari endpoints run past the 255 characters a default string + # column would give us. If this ever fails the migration has regressed. + def test_a_long_endpoint_is_stored_whole + long_endpoint = "https://updates.push.services.mozilla.com/wpush/v2/#{'a' * 300}" + + add_auth_header_for(user: @user) + post '/api/push_subscriptions', FactoryBot.attributes_for(:push_subscription, endpoint: long_endpoint) + + assert_equal 201, last_response.status + assert_equal long_endpoint, PushSubscription.last.endpoint + end + + def test_deleting_the_user_deletes_their_registrations + FactoryBot.create(:push_subscription, user: @user) + + assert_difference 'PushSubscription.count', -1 do + @user.destroy! + end + end +end diff --git a/test/api/settings_push_test.rb b/test/api/settings_push_test.rb new file mode 100644 index 0000000000..7a37f587ef --- /dev/null +++ b/test/api/settings_push_test.rb @@ -0,0 +1,97 @@ +require 'test_helper' + +# MN-C01: the authenticated front end reads the VAPID public key from +# /api/settings so it is configured in one place instead of being copied into +# the web repo and going stale the first time the keys are rotated. +# +# The endpoint is intentionally authenticated because it also reports protected +# feature flags. The anonymous case at the bottom pins down that neither VAPID +# key can leak when no credentials are supplied. +# +# Separate from settings_test.rb on purpose. That file predates rubocop's style +# rules and already carries 17 offenses; adding to it would either add more or +# mean reformatting a file this ticket has no business touching. +class SettingsPushTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + + def app + Rails.application + end + + setup do + clear_auth_header + add_auth_header_for + end + + teardown do + clear_auth_header + end + + # Restores whatever was there rather than deleting. The development container + # really does have these set, so a test that assumed they were absent would + # pass in CI and fail on a developer's machine. + def with_env(values) + previous = values.keys.index_with { |key| ENV.fetch(key, nil) } + values.each { |key, value| value.nil? ? ENV.delete(key) : ENV[key] = value } + yield + ensure + previous.each { |key, value| value.nil? ? ENV.delete(key) : ENV[key] = value } + end + + def with_vapid_keys(&) + with_env({ 'DOUBTFIRE_VAPID_PUBLIC_KEY' => 'BTestPublicKey', 'DOUBTFIRE_VAPID_PRIVATE_KEY' => 'BTestPrivateKey' }, &) + end + + def without_vapid_keys(&) + with_env({ 'DOUBTFIRE_VAPID_PUBLIC_KEY' => nil, 'DOUBTFIRE_VAPID_PRIVATE_KEY' => nil }, &) + end + + def test_the_public_key_is_published_when_push_is_configured + with_vapid_keys do + get '/api/settings' + + assert_equal 200, last_response.status + assert_equal true, last_response_body['pushEnabled'] + assert_equal 'BTestPublicKey', last_response_body['vapidPublicKey'] + end + end + + # Without keys the client must not offer the opt-in. Subscribing would fail in + # the browser with nothing on screen to explain why. + def test_push_is_reported_unavailable_without_keys + without_vapid_keys do + get '/api/settings' + + assert_equal 200, last_response.status + assert_equal false, last_response_body['pushEnabled'] + assert_nil last_response_body['vapidPublicKey'] + end + end + + # The browser needs the public key after sign-in. The private key must never + # be included in the settings response. + def test_the_private_key_is_never_published + with_vapid_keys do + get '/api/settings' + + assert_not_includes last_response.body, 'BTestPrivateKey' + assert_not_includes last_response_body.keys, 'vapidPrivateKey' + end + end + + def test_push_settings_are_not_published_to_an_anonymous_caller + clear_auth_header + + with_vapid_keys do + get '/api/settings' + + assert_equal 419, last_response.status + assert_not_includes last_response.body, 'BTestPublicKey' + assert_not_includes last_response.body, 'BTestPrivateKey' + assert_not_includes last_response_body.keys, 'vapidPublicKey' + assert_not_includes last_response_body.keys, 'vapidPrivateKey' + end + end +end diff --git a/test/api/units/task_definitions_api_test.rb b/test/api/units/task_definitions_api_test.rb index 00967f32b0..c6ed25ca8d 100644 --- a/test/api/units/task_definitions_api_test.rb +++ b/test/api/units/task_definitions_api_test.rb @@ -1,4 +1,5 @@ require 'test_helper' +require 'minitest/mock' class TaskDefinitionsTest < ActiveSupport::TestCase include Rack::Test::Methods @@ -35,21 +36,21 @@ def test_task_definition_cud data_to_post = { task_def: { - tutorial_stream_abbr: unit.tutorial_streams.first.abbreviation, - name: 'New Task Def', - description: 'First task def', - weighting: 4, - target_grade: 1, - group_set_id: unit.group_sets.first.id, - start_date: unit.start_date, - target_date: unit.start_date + 7.days, - due_date: unit.start_date + 21.days, - abbreviation: 'P1.1', - restrict_status_updates: false, - upload_requirements: '[ { "key": "file0", "name": "Shape Class", "type": "document" } ]', - plagiarism_warn_pct: 80, - is_graded: false, - max_quality_pts: 0 + tutorial_stream_abbr: unit.tutorial_streams.first.abbreviation, + name: 'New Task Def', + description: 'First task def', + weighting: 4, + target_grade: 1, + group_set_id: unit.group_sets.first.id, + start_date: unit.start_date, + target_date: unit.start_date + 7.days, + due_date: unit.start_date + 21.days, + abbreviation: 'P1.1', + restrict_status_updates: false, + upload_requirements: '[ { "key": "file0", "name": "Shape Class", "type": "document" } ]', + plagiarism_warn_pct: 80, + is_graded: false, + max_quality_pts: 0 } } @@ -69,21 +70,21 @@ def test_task_definition_cud data_to_put = { task_def: { - tutorial_stream_abbr: unit.tutorial_streams.last.abbreviation, - name: 'New Task Def 1', - description: 'First task def 1', - weighting: 2, - target_grade: 2, - group_set_id: nil, - start_date: unit.start_date + 2.days, - target_date: unit.start_date + 9.days, - due_date: unit.start_date + 23.days, - abbreviation: 'P1.2', - restrict_status_updates: true, - upload_requirements: [ { "key": "file0", "name": "Other Class", "type": "document" } ].to_json, - plagiarism_warn_pct: 80, - is_graded: false, - max_quality_pts: 0 + tutorial_stream_abbr: unit.tutorial_streams.last.abbreviation, + name: 'New Task Def 1', + description: 'First task def 1', + weighting: 2, + target_grade: 2, + group_set_id: nil, + start_date: unit.start_date + 2.days, + target_date: unit.start_date + 9.days, + due_date: unit.start_date + 23.days, + abbreviation: 'P1.2', + restrict_status_updates: true, + upload_requirements: [{ "key": "file0", "name": "Other Class", "type": "document" }].to_json, + plagiarism_warn_pct: 80, + is_graded: false, + max_quality_pts: 0 } } @@ -101,6 +102,83 @@ def test_task_definition_cud assert_equal 2, td.weighting end + def new_task_definition_payload(unit) + { + task_def: { + name: 'Notification Queue Test', + description: 'Task used to test notification queue behaviour', + weighting: 1, + target_grade: 1, + start_date: unit.start_date, + target_date: unit.start_date + 7.days, + due_date: unit.start_date + 14.days, + abbreviation: "QUEUE#{SecureRandom.hex(3)}", + restrict_status_updates: false, + plagiarism_warn_pct: 80, + is_graded: false, + max_quality_pts: 0 + } + } + end + + def test_task_definition_creation_enqueues_new_task_notification + unit = FactoryBot.create(:unit, task_count: 0) + enqueued_task_definition_id = nil + + enqueue = lambda do |task_definition_id| + enqueued_task_definition_id = task_definition_id + end + + NewTaskAvailableNotificationJob.stub(:perform_async, enqueue) do + add_auth_header_for(user: unit.main_convenor_user) + + post_json( + "/api/units/#{unit.id}/task_definitions", + new_task_definition_payload(unit) + ) + end + + assert_equal 201, last_response.status, last_response_body + + created_task_definition = unit.task_definitions.order(:id).last + + assert_equal( + created_task_definition.id, + enqueued_task_definition_id + ) + assert_not_nil created_task_definition.reload.new_task_notifications_from + end + + def test_task_definition_creation_succeeds_when_enqueue_fails + unit = FactoryBot.create(:unit, task_count: 0) + + enqueue_failure = lambda do |_task_definition_id| + raise StandardError, 'Redis unavailable' + end + + NewTaskAvailableNotificationJob.stub( + :perform_async, + enqueue_failure + ) do + add_auth_header_for(user: unit.main_convenor_user) + + assert_difference('TaskDefinition.count', 1) do + post_json( + "/api/units/#{unit.id}/task_definitions", + new_task_definition_payload(unit) + ) + end + end + + assert_equal 201, last_response.status, last_response_body + created_task_definition = unit.task_definitions.order(:id).last + assert_equal( + 'Notification Queue Test', + created_task_definition.name + ) + assert_not_nil created_task_definition.new_task_notifications_from + end + def test_post_invalid_file_tasksheet test_unit = FactoryBot.create(:unit, task_count: 1) test_task_definition_id = test_unit.task_definitions.first.id @@ -134,6 +212,19 @@ def test_post_tasksheet assert test_task_definition.task_sheet assert_equal File.size(data_to_post[:file]), File.size(TaskDefinition.first.task_sheet) + + expected_filename = test_task_definition.task_sheet_filename + represented = Entities::TaskDefinitionEntity.represent(test_task_definition).as_json + assert_equal expected_filename, represented[:task_sheet_filename] + + get "/api/units/#{test_unit.id}/task_definitions/#{test_task_definition.id}/task_pdf" + assert_equal 200, last_response.status + inline_bytes = last_response.body + + get "/api/units/#{test_unit.id}/task_definitions/#{test_task_definition.id}/task_pdf", as_attachment: true + assert_equal 200, last_response.status + assert_equal inline_bytes, last_response.body + assert_equal %(attachment; filename="#{expected_filename}"), last_response.headers['Content-Disposition'] end def test_post_task_resources @@ -181,30 +272,30 @@ def test_post_task_resources ] # Save will trigger TII integration - create_tii_group_stub = stub_request(:put, %r[https://localhost/api/v1/groups/.*]). - with(tii_headers). - with(body: %r[.*id.*.*name.*type.*ASSIGNMENT.*group_context.*id.*name.*due_date.*report_generation.*IMMEDIATELY_AND_DUE_DATE.*]). - to_return(status: 200, body: "", headers: {}) - - post_attachment_stub = stub_request(:post, %r[https://localhost/api/v1/groups/.*/attachments]). - with(tii_headers). - with(body: "{\"title\":\"TestWordDoc.docx\",\"template\":false}"). - to_return( - status: 200, - body: TCAClient::AddGroupAttachmentResponse.new( - id: SecureRandom.uuid - ).to_json, - headers: {} - ) - - upload_stub = stub_request(:put, %r[https://localhost/api/v1/groups/.*/attachments/.*/original]). - with(tii_headers). - with(headers: {'Content-Type'=>'binary/octet-stream'}). - to_return(status: 200, body: '{ "message": "Successfully uploaded file for attachment ..." }', headers: {}) - - delete_stub = stub_request(:delete, %r[https://localhost/api/v1/groups/.*/attachments/.*]). - with(tii_headers). - to_return(status: 200, body: "", headers: {}) + create_tii_group_stub = stub_request(:put, %r[https://localhost/api/v1/groups/.*]) + .with(tii_headers) + .with(body: %r[.*id.*.*name.*type.*ASSIGNMENT.*group_context.*id.*name.*due_date.*report_generation.*IMMEDIATELY_AND_DUE_DATE.*]) + .to_return(status: 200, body: "", headers: {}) + + post_attachment_stub = stub_request(:post, %r[https://localhost/api/v1/groups/.*/attachments]) + .with(tii_headers) + .with(body: "{\"title\":\"TestWordDoc.docx\",\"template\":false}") + .to_return( + status: 200, + body: TCAClient::AddGroupAttachmentResponse.new( + id: SecureRandom.uuid + ).to_json, + headers: {} + ) + + upload_stub = stub_request(:put, %r[https://localhost/api/v1/groups/.*/attachments/.*/original]) + .with(tii_headers) + .with(headers: { 'Content-Type' => 'binary/octet-stream' }) + .to_return(status: 200, body: '{ "message": "Successfully uploaded file for attachment ..." }', headers: {}) + + delete_stub = stub_request(:delete, %r[https://localhost/api/v1/groups/.*/attachments/.*]) + .with(tii_headers) + .to_return(status: 200, body: "", headers: {}) td.save! @@ -241,21 +332,21 @@ def test_post_scorm def test_submission_creates_folders unit = Unit.first td = TaskDefinition.new({ - unit_id: unit.id, - tutorial_stream: unit.tutorial_streams.first, - name: 'test_submission_creates_folders', - description: 'test def', - weighting: 4, - target_grade: 0, - start_date: unit.start_date + 1.week, - target_date: unit.start_date + 2.weeks, - abbreviation: 'test_submission_creates_folders', - restrict_status_updates: false, - upload_requirements: [ { "key" => "file0", "name" => "Shape Class", "type" => "document" } ], - plagiarism_warn_pct: 0.8, - is_graded: false, - max_quality_pts: 0 - }) + unit_id: unit.id, + tutorial_stream: unit.tutorial_streams.first, + name: 'test_submission_creates_folders', + description: 'test def', + weighting: 4, + target_grade: 0, + start_date: unit.start_date + 1.week, + target_date: unit.start_date + 2.weeks, + abbreviation: 'test_submission_creates_folders', + restrict_status_updates: false, + upload_requirements: [{ "key" => "file0", "name" => "Shape Class", "type" => "document" }], + plagiarism_warn_pct: 0.8, + is_graded: false, + max_quality_pts: 0 + }) td.save! data_to_post = { @@ -295,21 +386,21 @@ def test_submission_creates_folders def test_change_to_group_after_submissions unit = Unit.first td = TaskDefinition.new({ - unit_id: unit.id, - tutorial_stream: unit.tutorial_streams.first, - name: 'Task to switch from ind to group after submission', - description: 'test def', - weighting: 4, - target_grade: 0, - start_date: unit.start_date + 1.week, - target_date: unit.start_date + 2.weeks, - abbreviation: 'TaskSwitchIndGrp', - restrict_status_updates: false, - upload_requirements: [ { "key" => 'file0', "name" => 'Shape Class', "type" => 'document' } ], - plagiarism_warn_pct: 0.8, - is_graded: false, - max_quality_pts: 0 - }) + unit_id: unit.id, + tutorial_stream: unit.tutorial_streams.first, + name: 'Task to switch from ind to group after submission', + description: 'test def', + weighting: 4, + target_grade: 0, + start_date: unit.start_date + 1.week, + target_date: unit.start_date + 2.weeks, + abbreviation: 'TaskSwitchIndGrp', + restrict_status_updates: false, + upload_requirements: [{ "key" => 'file0', "name" => 'Shape Class', "type" => 'document' }], + plagiarism_warn_pct: 0.8, + is_graded: false, + max_quality_pts: 0 + }) td.save! data_to_post = { @@ -335,7 +426,7 @@ def test_change_to_group_after_submissions # Change it to a group task - group_set = GroupSet.create!({name: 'test group set', unit: unit}) + group_set = GroupSet.create!({ name: 'test group set', unit: unit }) group_set.save! td.group_set = group_set @@ -836,8 +927,8 @@ def test_task_related_to_task_def_when_multiple_projects_tasks_and_tutorials end def test_change_draft_learning_summary_upload_requirements - unit = FactoryBot.create :unit, student_count:1, task_count:0 - upload_reqs = [{'key' => 'file0','name' => 'Draft learning summary','type' => 'document'}] + unit = FactoryBot.create :unit, student_count: 1, task_count: 0 + upload_reqs = [{ 'key' => 'file0', 'name' => 'Draft learning summary', 'type' => 'document' }] task_def = FactoryBot.create(:task_definition, unit: unit, upload_requirements: upload_reqs) # Set draft learning summary task defintion @@ -857,7 +948,7 @@ def test_change_draft_learning_summary_upload_requirements # Test change upload requirements to a non-document upload data_to_put = { task_def: { - upload_requirements: [{"key": "file0","name": "Code file","type": "code"}].to_json + upload_requirements: [{ "key": "file0", "name": "Code file", "type": "code" }].to_json } } @@ -984,4 +1075,93 @@ def test_download_student_submission_jobs end end end + + def test_due_date_update_enqueues_notification_job + unit = FactoryBot.create(:unit, task_count: 1) + task_def = unit.task_definitions.first + + previous_due_date = task_def[:due_date]&.to_date&.iso8601 + new_due_date = (task_def.due_date + 1.week).to_date + + data_to_put = { + task_def: { + due_date: new_due_date + } + } + + add_auth_header_for(user: unit.main_convenor_user) + + assert_difference( + -> { TaskDueDateChangedNotificationJob.jobs.size }, + 1 + ) do + put_json( + "/api/units/#{unit.id}/task_definitions/#{task_def.id}", + data_to_put + ) + end + + assert_equal 200, last_response.status, last_response_body + + assert_equal( + [ + task_def.id, + previous_due_date, + new_due_date.iso8601 + ], + TaskDueDateChangedNotificationJob.jobs.last['args'].first(3) + ) + assert_match(/\A[0-9a-f-]{36}\z/, TaskDueDateChangedNotificationJob.jobs.last['args'][3]) + end + + def test_unrelated_update_does_not_enqueue_due_date_notification_job + unit = FactoryBot.create(:unit, task_count: 1) + task_def = unit.task_definitions.first + + data_to_put = { + task_def: { + description: 'Updated without moving the due date.' + } + } + + add_auth_header_for(user: unit.main_convenor_user) + + assert_no_difference( + -> { TaskDueDateChangedNotificationJob.jobs.size } + ) do + put_json( + "/api/units/#{unit.id}/task_definitions/#{task_def.id}", + data_to_put + ) + end + + assert_equal 200, last_response.status, last_response_body + end + + def test_due_date_update_succeeds_when_enqueue_fails + unit = FactoryBot.create(:unit, task_count: 1) + task_def = unit.task_definitions.first + new_due_date = (task_def.due_date + 1.week).to_date + + data_to_put = { + task_def: { + due_date: new_due_date + } + } + + add_auth_header_for(user: unit.main_convenor_user) + + TaskDueDateChangedNotificationJob.stub( + :perform_async, + ->(*) { raise StandardError, 'Redis unavailable' } + ) do + put_json( + "/api/units/#{unit.id}/task_definitions/#{task_def.id}", + data_to_put + ) + end + + assert_equal 200, last_response.status, last_response_body + assert_equal new_due_date, task_def.reload[:due_date].to_date + end end diff --git a/test/config/vapid_configuration_test.rb b/test/config/vapid_configuration_test.rb new file mode 100644 index 0000000000..ac01e5b355 --- /dev/null +++ b/test/config/vapid_configuration_test.rb @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +require 'test_helper' + +class VapidConfigurationTest < ActiveSupport::TestCase + def test_development_can_omit_push_but_production_cannot + assert_nil VapidConfiguration.validate!(production: false, environment: {}) + error = assert_raises(ArgumentError) { VapidConfiguration.validate!(production: true, environment: {}) } + assert_includes error.message, 'DOUBTFIRE_VAPID_PRIVATE_KEY' + end + + def test_matching_keys_pass_and_mismatched_keys_fail_without_disclosing_values + key = WebPush.generate_key + environment = { 'DOUBTFIRE_VAPID_PUBLIC_KEY' => key.public_key, 'DOUBTFIRE_VAPID_PRIVATE_KEY' => key.private_key } + VapidConfiguration.validate!(production: true, environment: environment) + environment['DOUBTFIRE_VAPID_PRIVATE_KEY'] = WebPush.generate_key.private_key + error = assert_raises(ArgumentError) { VapidConfiguration.validate!(production: true, environment: environment) } + environment.each_value { |value| assert_not_includes error.message, value } + assert_nil error.cause + end + + def test_malformed_keys_fail_without_disclosing_values + error = assert_raises(ArgumentError) do + VapidConfiguration.validate!(production: true, environment: { + 'DOUBTFIRE_VAPID_PUBLIC_KEY' => 'malformed-public', 'DOUBTFIRE_VAPID_PRIVATE_KEY' => 'secret-marker' + }) + end + assert_not_includes error.message, 'secret-marker' + end +end diff --git a/test/factories/notification_factory.rb b/test/factories/notification_factory.rb new file mode 100644 index 0000000000..13875c9fc6 --- /dev/null +++ b/test/factories/notification_factory.rb @@ -0,0 +1,44 @@ +require 'faker' + +FactoryBot.define do + factory :notification do + user + notification_type { 'general' } + event { "#{notification_type}_event" } + message { Faker::Lorem.sentence } + link { nil } + read_at { nil } + + trait :task do + notification_type { 'task' } + end + + trait :feedback do + notification_type { 'feedback' } + end + + trait :portfolio do + notification_type { 'portfolio' } + end + + trait :extension do + notification_type { 'extension' } + end + + trait :general do + notification_type { 'general' } + end + + trait :unit_hub do + notification_type { 'unit_hub' } + end + + trait :read do + read_at { Time.zone.now } + end + + trait :unread do + read_at { nil } + end + end +end diff --git a/test/factories/push_subscriptions_factory.rb b/test/factories/push_subscriptions_factory.rb new file mode 100644 index 0000000000..9b4a87c8c1 --- /dev/null +++ b/test/factories/push_subscriptions_factory.rb @@ -0,0 +1,18 @@ +FactoryBot.define do + factory :push_subscription do + user + + # The endpoint is unique across the whole table, not per user, so the + # sequence alone is not enough. A test that leaks a row past its transaction + # would leave that endpoint in the database for the next run and the + # collision would look like a bug in whatever test built it second. + sequence(:endpoint) { |n| "https://fcm.googleapis.com/fcm/send/factory-#{n}-#{SecureRandom.hex(8)}" } + + # A throwaway browser key pair. The p256dh has to be a real prime256v1 + # public key because PushNotificationService encrypts against it through the + # web-push gem, and the gem cannot encrypt to a made up string. The auth + # secret has to decode to 16 bytes for the same reason. + p256dh { 'BJy8RpjMkwOPDIIXSu-FTe7OosAwY9G86_evhrn0jJbPnoxXjBYpn7aPHEIaRh3GxCzFvwYXjKWvtu3FEMaBQMY=' } + auth { 'CUkmaYqq8eINt1HTnFY65w==' } + end +end diff --git a/test/helpers/push_notification_helper.rb b/test/helpers/push_notification_helper.rb new file mode 100644 index 0000000000..903e88d06f --- /dev/null +++ b/test/helpers/push_notification_helper.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +module TestHelpers + module PushNotificationHelper + def parsed_push_notification(notification) + JSON.parse( + PushNotificationService.payload_for(notification) + ).fetch('notification') + end + + def assert_valid_push_payload(notification, expected_link:, expected_body: nil) + push = parsed_push_notification(notification) + data = push.fetch('data') + expected_body ||= notification.message.to_s.truncate( + PushNotificationService::MAX_BODY_LENGTH + ) + + assert push['title'].present?, 'push title must be present' + assert_equal expected_body, push['body'] + assert_operator( + push['body'].length, + :<=, + PushNotificationService::MAX_BODY_LENGTH + ) + assert_equal notification.id, data['notification_id'] + assert_equal expected_link, data['link'] + assert data['link'].present?, 'push data.link must be present' + + push + end + end +end diff --git a/test/mailers/address_with_name_test.rb b/test/mailers/address_with_name_test.rb new file mode 100644 index 0000000000..56a182cca7 --- /dev/null +++ b/test/mailers/address_with_name_test.rb @@ -0,0 +1,56 @@ +require 'test_helper' + +# User#name is built from first_name/last_name, which are user-editable and +# validated for presence only. Interpolating it straight into +# %("#{name}" <#{email}>) let a name carrying a quote close the display name and +# add a second recipient, and a name carrying a newline fold an extra header +# into the message. +class AddressWithNameTest < ActiveSupport::TestCase + class Probe < ApplicationMailer + def check(user) + address_with_name(user) + end + end + + Person = Struct.new(:name, :email) + + def addresses_in(value) + Mail::AddressList.new(value).addresses + end + + def assert_single_recipient(name) + built = Probe.new.check(Person.new(name, 'ann@uni.test')) + + refute_match(/[\r\n]/, built, 'a display name must not be able to fold a header') + parsed = addresses_in(built) + assert_equal 1, parsed.size, "expected one recipient, got #{parsed.map(&:address).join(', ')}" + assert_equal 'ann@uni.test', parsed.first.address + end + + def test_a_quote_in_the_name_cannot_add_a_second_recipient + assert_single_recipient('Ann " <evil@attacker.test>, "x') + end + + def test_a_comma_in_the_name_cannot_add_a_second_recipient + assert_single_recipient('Ann, evil@attacker.test') + end + + def test_a_newline_in_the_name_cannot_fold_a_header + assert_single_recipient("Ann\r\nBcc: evil@attacker.test") + assert_single_recipient("Ann\nBcc: evil@attacker.test") + end + + def test_an_ordinary_name_is_left_readable + built = Probe.new.check(Person.new('Ann Example', 'ann@uni.test')) + assert_equal 'Ann Example <ann@uni.test>', built + end + + # The point of moving it to ApplicationMailer: every mailer that addresses a + # person gets it, not just the one that had it. + def test_every_mailer_can_reach_it + [NotificationsMailer, PortfolioEvidenceMailer, TutorNoteMailer].each do |mailer| + assert mailer.private_method_defined?(:address_with_name), + "#{mailer} cannot reach address_with_name" + end + end +end diff --git a/test/mailers/azure_smtp_sender_test.rb b/test/mailers/azure_smtp_sender_test.rb new file mode 100644 index 0000000000..ff1b5f9d5b --- /dev/null +++ b/test/mailers/azure_smtp_sender_test.rb @@ -0,0 +1,71 @@ +require 'test_helper' + +class AzureSmtpSenderTest < ActionMailer::TestCase + HUMAN_SENDER = 'Tutor Example <tutor@example.edu>'.freeze + VERIFIED_SENDER = 'OnTrack <noreply@ontrack.example>'.freeze + Sender = Struct.new(:name) + + def test_non_production_communication_keeps_existing_from_header + mail = communication_email + + assert_equal ['tutor@example.edu'], mail.from + assert_nil mail.reply_to + end + + def test_production_communication_uses_verified_from_and_human_reply_to + with_production_sender do + mail = communication_email + + assert_equal ['noreply@ontrack.example'], mail.from + assert_equal ['tutor@example.edu'], mail.reply_to + end + end + + def test_production_system_mail_does_not_add_misleading_reply_to + previous_error_recipient = Doubtfire::Application.config.email_errors_to + Doubtfire::Application.config.email_errors_to = 'Operations <operations@example.edu>' + + with_production_sender do + mail = ErrorLogMailer.error_message('test', 'test message', StandardError.new('test error')) + + assert_equal ['noreply@ontrack.example'], mail.from + assert_nil mail.reply_to + end + ensure + Doubtfire::Application.config.email_errors_to = previous_error_recipient + end + + def test_production_mail_fails_closed_without_configured_sender + with_production_sender(nil) do + error = assert_raises(ArgumentError) { communication_email.message } + + assert_equal 'institution email_sender must be configured in production', error.message + end + end + + private + + def communication_email + CommunicationsMailer.communication_email( + to: 'Student Example <student@example.edu>', + from: HUMAN_SENDER, + subject: 'Test communication', + body: 'Test body', + recipient: nil, + sender: Sender.new('Tutor Example'), + unit: nil, + rule: nil + ) + end + + def with_production_sender(sender = VERIFIED_SENDER, &) + institution = Doubtfire::Application.config.institution + previous_sender = institution[:email_sender] + production = ActiveSupport::EnvironmentInquirer.new('production') + institution[:email_sender] = sender + + Rails.stub(:env, production, &) + ensure + institution[:email_sender] = previous_sender + end +end diff --git a/test/mailers/communications_mailer_test.rb b/test/mailers/communications_mailer_test.rb new file mode 100644 index 0000000000..dafa275c6f --- /dev/null +++ b/test/mailers/communications_mailer_test.rb @@ -0,0 +1,81 @@ +require 'test_helper' + +class CommunicationsMailerTest < ActionMailer::TestCase + # Regression for the nested-ERB-comment leak: the text part used + # `<%# Hi <%= ... %> %>`, and because an ERB comment ends at the first `%>` + # the trailing ` %>` printed literally in every email. Assert the rendered + # text part carries no raw ERB delimiter and the greeting and sign-off render. + def test_text_part_renders_without_leaking_erb + unit = FactoryBot.create :unit + recipient = FactoryBot.create :user + sender = FactoryBot.create :user, :convenor + + mail = CommunicationsMailer.communication_email( + to: recipient.email, + from: sender.email, + subject: 'Weekly update', + body: "First paragraph.\nSecond paragraph.", + recipient: recipient, + sender: sender, + unit: unit, + rule: nil + ) + + text = mail.text_part.body.to_s + + assert_not_includes text, '%>', "text part leaked a raw ERB delimiter:\n#{text}" + assert_not_includes text, '<%', "text part leaked a raw ERB delimiter:\n#{text}" + assert_includes text, "Hi #{recipient.first_name}," + assert_includes text, 'First paragraph.' + assert_includes text, 'Cheers,' + assert_includes text, "on behalf of #{sender.name}" + ensure + unit&.destroy! + end + + def test_communication_email_has_greeting_sign_off_and_unsubscribe + recipient = FactoryBot.create :user, :student + sender = FactoryBot.create :user, :convenor + + mail = CommunicationsMailer.communication_email( + to: recipient.email, + from: sender.email, + subject: 'A message from your convenor', + body: "First paragraph.\nSecond paragraph.", + recipient: recipient, + sender: sender, + unit: nil, + rule: nil + ) + + html = mail.html_part.body.to_s + expected_name = recipient.nickname.presence || recipient.first_name + + assert_includes html, "Hi #{ERB::Util.html_escape(expected_name)}," + assert_includes html, "on behalf of #{ERB::Util.html_escape(sender.name)}" + assert_includes html, 'Unsubscribe' + assert_includes html, 'First paragraph.' + assert_includes html, 'Second paragraph.' + end + + def test_communication_email_html_escapes_apostrophes_in_names + recipient = FactoryBot.create :user, :student, first_name: "Sha'Carri", nickname: nil + sender = FactoryBot.create :user, :convenor, first_name: 'Antonio', last_name: "D'Amore" + + mail = CommunicationsMailer.communication_email( + to: recipient.email, + from: sender.email, + subject: 'A message from your convenor', + body: 'A message.', + recipient: recipient, + sender: sender, + unit: nil, + rule: nil + ) + + html = mail.html_part.body.to_s + + assert_includes html, 'Hi Sha'Carri,' + assert_includes html, 'on behalf of Antonio D'Amore' + end +end diff --git a/test/mailers/notifications_mailer_test.rb b/test/mailers/notifications_mailer_test.rb new file mode 100644 index 0000000000..7ee05274e7 --- /dev/null +++ b/test/mailers/notifications_mailer_test.rb @@ -0,0 +1,222 @@ +require 'test_helper' + +# EN-T04: every event's mailer templates render without raising, in both +# HTML and text. +# +# NOTE FOR FUTURE CONTRIBUTORS: this file does not discover events +# automatically. When you add a new event, add its name and notification +# type to the EVENTS hash below. A missing entry here means a broken or +# missing template for that event falls back silently to +# single_notification and nothing catches it. +class NotificationsMailerTest < ActionMailer::TestCase + # event => notification_type, matching the eleven non-demo events + # currently wired through NotificationService across the app. + EVENTS = { + 'task_comment_created' => 'feedback', + 'task_status_changed' => 'task', + 'task_due_soon' => 'task', + 'task_due_date_changed' => 'task', + 'new_task_available' => 'task', + 'task_submitted' => 'task', + 'extension_assessed' => 'extension', + 'group_membership_changed' => 'general', + 'discussion_request_created' => 'feedback', + 'portfolio_received' => 'portfolio', + 'tutorial_changed' => 'general' + }.freeze + + LINK = '/projects/1/dashboard/A1'.freeze + + EVENTS.each do |event, notification_type| + define_method("test_#{event}_renders_html_and_text") do + # The mailer falls back to the generic template when an event-specific + # template is missing, so require both event-specific template files. + %w[html text].each do |format| + template_path = Rails.root.join( + 'app', + 'views', + 'notifications_mailer', + "#{event}.#{format}.erb" + ) + + assert template_path.file?, + "#{event}: missing event-specific #{format} template" + end + + notification = FactoryBot.create( + :notification, + notification_type: notification_type, + event: event, + message: "A realistic message for #{event}, long enough to catch interpolation errors.", + link: LINK + ) + + mail = NotificationsMailer.single_notification(notification) + + assert mail.html_part.body.to_s.present?, "#{event}: HTML part did not render" + assert mail.text_part.body.to_s.present?, "#{event}: text part did not render" + end + + define_method("test_#{event}_subject_is_event_specific") do + notification = FactoryBot.create( + :notification, + notification_type: notification_type, + event: event + ) + + mail = NotificationsMailer.single_notification(notification) + + product_name = Doubtfire::Application.config.institution[:product_name] + fallback_subject = "#{product_name}: New notification" + expected_subject = "#{product_name}: #{NotificationsMailer::SUBJECTS.fetch(event)}" + + assert mail.subject.present?, "#{event}: subject was blank" + assert_not_equal fallback_subject, mail.subject, + "#{event}: still used the generic fallback subject" + assert_equal expected_subject, mail.subject, + "#{event}: subject did not match its configured subject" + end + + define_method("test_#{event}_link_is_in_the_body") do + notification = FactoryBot.create( + :notification, + notification_type: notification_type, + event: event, + link: LINK + ) + + mail = NotificationsMailer.single_notification(notification) + expected_url = "#{Doubtfire::Application.config.institution[:host]}#{LINK}" + + assert_includes mail.html_part.body.to_s, expected_url, "#{event}: exact link missing from HTML body" + assert_includes mail.text_part.body.to_s, expected_url, "#{event}: exact link missing from text body" + end + end + + # Events that came after the subject map. The Unit Hub templates render + # without their announcement or session, so a bare notification is enough. + LATER_EVENTS = { + 'portfolio_submitted' => 'portfolio', + 'task_automated_comment_created' => 'feedback', + 'unit_announcement_published' => 'unit_hub', + 'unit_announcement_updated' => 'unit_hub', + 'unit_session_changed' => 'unit_hub', + 'unit_session_starting_soon' => 'unit_hub' + }.freeze + + LATER_EVENTS.each do |event, notification_type| + define_method("test_#{event}_subject_is_event_specific") do + notification = FactoryBot.create( + :notification, + notification_type: notification_type, + event: event + ) + + mail = NotificationsMailer.single_notification(notification) + product_name = Doubtfire::Application.config.institution[:product_name] + + assert_equal "#{product_name}: #{NotificationsMailer::SUBJECTS.fetch(event)}", mail.subject + assert mail.html_part.body.to_s.present?, "#{event}: HTML part did not render" + end + end + + def test_unknown_event_uses_generic_subject_fallback + notification = FactoryBot.create( + :notification, + notification_type: 'general', + event: 'unwired_event', + message: 'A generic notification.', + link: LINK + ) + + mail = NotificationsMailer.single_notification(notification) + + expected_subject = + "#{Doubtfire::Application.config.institution[:product_name]}: New notification" + + assert_equal expected_subject, mail.subject + end + + def test_configured_sender_is_used + institution = Doubtfire::Application.config.institution + previous_sender = institution[:email_sender] + institution[:email_sender] = 'notifications@example.edu' + + notification = FactoryBot.create( + :notification, + notification_type: 'feedback', + event: 'task_comment_created' + ) + + mail = NotificationsMailer.single_notification(notification) + + assert_equal ['notifications@example.edu'], mail.from + ensure + institution[:email_sender] = previous_sender + end + + # Security: the recipient address used to be built as %("#{user.name}" + # <#{user.email}>). User#name comes from the user-editable first_name, so a + # name containing a quote and a comma could break out of the display name and + # add a second recipient. The address is now built through Mail, which escapes + # the display name, so it can never inject another address. + def test_a_malicious_display_name_cannot_inject_a_second_recipient + user = FactoryBot.create(:user, :student, first_name: 'a",x@evil.com', last_name: 'Test') + notification = FactoryBot.create( + :notification, + user: user, + notification_type: 'feedback', + event: 'task_comment_created', + message: 'A comment arrived.' + ) + + mail = NotificationsMailer.single_notification(notification) + + assert_equal 1, mail.to.length, "expected exactly one recipient, got #{mail.to.inspect}" + assert_includes mail.to.map(&:downcase), user.email.downcase + assert_not_includes mail.to.join(','), 'evil.com' + end + + def test_additional_copy_is_a_separate_message_without_recipient_disclosure + notification = FactoryBot.create( + :notification, + notification_type: 'feedback', + event: 'task_comment_created' + ) + + mail = NotificationsMailer.additional_notification_copy( + notification, + 'secondary@example.org' + ) + + expected_subject = "#{Doubtfire::Application.config.institution[:product_name]}: #{NotificationsMailer::SUBJECTS.fetch('task_comment_created')}" + assert_equal expected_subject, mail.subject + assert_equal ['secondary@example.org'], mail.to + assert_empty mail.cc.to_a + assert_empty mail.bcc.to_a + assert_not_includes mail.header.to_s, notification.user.email + assert mail.html_part.body.to_s.present? + assert mail.text_part.body.to_s.present? + end + + # The same production sender rule as every other mailer on 11.0.x. + def test_additional_mail_refuses_an_unconfigured_sender_in_production + institution = Doubtfire::Application.config.institution + previous_sender = institution[:email_sender] + institution[:email_sender] = nil + user = FactoryBot.create(:user, email: 'primary@example.edu') + notification = FactoryBot.create(:notification, :feedback, user: user, event: 'task_comment_created') + record = AdditionalNotificationEmailService.request(user: user, email: 'secondary@example.org') + + Rails.stub(:env, ActiveSupport::EnvironmentInquirer.new('production')) do + assert_raises(ArgumentError) do + NotificationsMailer.additional_notification_copy(notification, 'secondary@example.org').message + end + assert_raises(ArgumentError) do + AdditionalNotificationEmailMailer.verification(record).message + end + end + ensure + institution[:email_sender] = previous_sender + end +end diff --git a/test/mailers/tutor_note_mailer_test.rb b/test/mailers/tutor_note_mailer_test.rb new file mode 100644 index 0000000000..351a92a9c6 --- /dev/null +++ b/test/mailers/tutor_note_mailer_test.rb @@ -0,0 +1,45 @@ +require 'test_helper' + +class TutorNoteMailerTest < ActionMailer::TestCase + # BGW-21: a tutor note can refer to a task that is absent from the filtered + # marking inbox. Send staff through the unfiltered task explorer instead of + # the student dashboard or a filtered inbox route that cannot select it. + def test_notify_tutor_note_links_to_task_explorer_when_task_is_not_in_inbox + unit = FactoryBot.create :unit + convenor = FactoryBot.create :user, :convenor + + ur = unit.employ_staff convenor, Role.convenor + unit.update main_convenor: ur + + tutor = FactoryBot.create :user, :tutor + tutor_role = unit.employ_staff tutor, Role.tutor + + project = unit.active_projects.first + task = project.task_for_task_definition(unit.task_definitions.first) + task.task_definition.update!(abbreviation: 'Portfolio Reflection') + task.update!(task_status: TaskStatus.not_started) + + assert_not_includes unit.tasks_for_task_inbox(convenor).map(&:task_id), task.id, + 'regression setup requires a task that the marking inbox filters out' + + tutor_note = TutorNote.create!( + unit_role: tutor_role, + user: tutor, + task: task, + note: 'Please take a look at this submission.', + read_by_unit_role: false + ) + + mail = TutorNoteMailer.notify_tutor_note(tutor_note, convenor) + + host = Doubtfire::Application.config.institution[:host] + expected = "#{host}/units/#{unit.id}/tasks/definition/#{project.user.id}/Portfolio%20Reflection" + + assert mail.html_part.body.to_s.include?(expected), 'html link should point at the task explorer' + assert mail.text_part.body.to_s.include?(expected), 'text link should point at the task explorer' + assert_not mail.html_part.body.to_s.include?('/tasks/inbox/'), 'the filtered inbox route should be gone' + assert_not mail.html_part.body.to_s.include?('?tutor=true'), "the dead tutor flag should be gone" + + unit.destroy! + end +end diff --git a/test/mailers/unit_mail_test.rb b/test/mailers/unit_mail_test.rb index c84b18a16d..3604a31efc 100644 --- a/test/mailers/unit_mail_test.rb +++ b/test/mailers/unit_mail_test.rb @@ -30,9 +30,12 @@ def test_send_portfolio_ready_from_main_convenor mail = PortfolioEvidenceMailer.portfolio_ready(project) - assert_equal 1, mail.from().count - assert_equal convenor.email, mail.from().first - assert mail.html_part.body.include? "projects/#{project.id}/portfolio" + assert_equal 1, mail.from.count + assert_equal convenor.email, mail.from.first + assert mail.html_part.body.include? "#{Doubtfire::Application.config.institution[:host]}/projects/#{project.id}/portfolio" + # The text-only part must carry the same deep link so plain-text recipients + # have something to act on (BGW-29). + assert mail.text_part.body.include?("projects/#{project.id}/portfolio"), mail.text_part.body.to_s unit.destroy! end @@ -48,9 +51,10 @@ def test_send_portfolio_fail_from_main_convenor mail = PortfolioEvidenceMailer.portfolio_failed(project) - assert_equal 1, mail.from().count - assert_equal convenor.email, mail.from().first - assert mail.html_part.body.include? "projects/#{project.id}/portfolio" + assert_equal 1, mail.from.count + assert_equal convenor.email, mail.from.first + assert mail.html_part.body.include? "#{Doubtfire::Application.config.institution[:host]}/projects/#{project.id}/portfolio" + assert mail.text_part.body.include?("projects/#{project.id}/portfolio"), mail.text_part.body.to_s unit.destroy! end @@ -64,13 +68,72 @@ def test_send_overseer_assessment_failed_email project = unit.active_projects.first task = project.task_for_task_definition(unit.task_definitions.first) + task.task_definition.update!(abbreviation: 'Portfolio Reflection') mail = PortfolioEvidenceMailer.overseer_assessment_failed(project, [task]) assert_equal 1, mail.from.count assert_equal convenor.email, mail.from.first assert_equal project.student.email, mail.to.first - assert mail.html_part.body.include? "projects/#{project.id}/dashboard/#{task.task_definition.abbreviation}" + assert mail.html_part.body.include? "#{Doubtfire::Application.config.institution[:host]}/projects/#{project.id}/dashboard/Portfolio%20Reflection" + assert mail.text_part.body.include?("projects/#{project.id}/dashboard/Portfolio%20Reflection"), mail.text_part.body.to_s + end + + def test_send_task_pdf_failed_email_carries_a_deep_link + unit = FactoryBot.create :unit + convenor = FactoryBot.create :user, :convenor + ur = unit.employ_staff convenor, Role.convenor + unit.update main_convenor: ur + + project = unit.active_projects.first + task = project.task_for_task_definition(unit.task_definitions.first) + task.task_definition.update!(abbreviation: 'Portfolio Reflection') + + mail = PortfolioEvidenceMailer.task_pdf_failed(project, [task]) + + host = Doubtfire::Application.config.institution[:host] + link = "#{host}/projects/#{project.id}/dashboard/Portfolio%20Reflection" + assert mail.html_part.body.include?(link), mail.html_part.body.to_s + assert mail.text_part.body.include?("projects/#{project.id}/dashboard/Portfolio%20Reflection"), mail.text_part.body.to_s + unit.destroy! + end + + def test_send_task_pdf_ready_email_carries_a_deep_link + unit = FactoryBot.create :unit + convenor = FactoryBot.create :user, :convenor + ur = unit.employ_staff convenor, Role.convenor + unit.update main_convenor: ur + + project = unit.active_projects.first + task = project.task_for_task_definition(unit.task_definitions.first) + task.task_definition.update!(abbreviation: 'Portfolio Reflection') + + mail = PortfolioEvidenceMailer.task_pdf_ready_message(project, [task]) + + host = Doubtfire::Application.config.institution[:host] + link = "#{host}/projects/#{project.id}/dashboard/Portfolio%20Reflection" + assert mail.html_part.body.include?(link), mail.html_part.body.to_s + assert mail.text_part.body.include?("projects/#{project.id}/dashboard/Portfolio%20Reflection"), mail.text_part.body.to_s + unit.destroy! + end + + def test_send_task_feedback_ready_email_carries_a_deep_link + unit = FactoryBot.create :unit + convenor = FactoryBot.create :user, :convenor + ur = unit.employ_staff convenor, Role.convenor + unit.update main_convenor: ur + + project = unit.active_projects.first + task = project.task_for_task_definition(unit.task_definitions.first) + task.task_definition.update!(abbreviation: 'Portfolio Reflection') + + mail = PortfolioEvidenceMailer.task_feedback_ready(project, [task]) + + host = Doubtfire::Application.config.institution[:host] + link = "#{host}/projects/#{project.id}/dashboard/Portfolio%20Reflection" + assert mail.html_part.body.include?(link), mail.html_part.body.to_s + assert mail.text_part.body.include?("projects/#{project.id}/dashboard/Portfolio%20Reflection"), mail.text_part.body.to_s + unit.destroy! end end diff --git a/test/models/additional_notification_email_test.rb b/test/models/additional_notification_email_test.rb new file mode 100644 index 0000000000..42464cf811 --- /dev/null +++ b/test/models/additional_notification_email_test.rb @@ -0,0 +1,140 @@ +require 'test_helper' + +class AdditionalNotificationEmailTest < ActiveSupport::TestCase + setup do + @user = FactoryBot.create(:user, email: 'primary@example.edu') + end + + def test_normalises_the_address_and_never_stores_a_verification_token + record = AdditionalNotificationEmailService.request( + user: @user, + email: ' SECONDARY@Example.ORG ' + ) + + assert_equal 'secondary@example.org', record.email + assert_not record.verified? + assert_not record.attributes.key?('verification_token') + assert_not record.attributes.key?('verification_token_digest') + assert_equal [record.id, record.verification_version], + AdditionalNotificationEmailVerificationJob.jobs.last['args'] + end + + def test_primary_address_cannot_be_reused_case_insensitively + error = assert_raises(ActiveRecord::RecordInvalid) do + AdditionalNotificationEmailService.request( + user: @user, + email: 'PRIMARY@example.edu' + ) + end + + assert_includes error.message, 'must be different from the institutional email' + end + + def test_verification_is_required_and_a_link_cannot_be_replayed + record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + token = record.verification_token + + verified = AdditionalNotificationEmailService.verify(token: token) + + assert verified.reload.verified? + assert_raises(AdditionalNotificationEmailService::AlreadyVerified) do + AdditionalNotificationEmailService.verify(token: token) + end + end + + def test_expired_token_is_rejected + record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + token = record.verification_token + # Deliberately bypass the model to exercise a token that expired after issue. + record.update_column(:verification_expires_at, 1.minute.ago) # rubocop:disable Rails/SkipsModelValidations + + assert_raises(AdditionalNotificationEmailService::InvalidToken) do + AdditionalNotificationEmailService.verify(token: token) + end + end + + def test_replacement_invalidates_the_previous_link + first = AdditionalNotificationEmailService.request( + user: @user, + email: 'first@example.org' + ) + old_token = first.verification_token + + replacement = AdditionalNotificationEmailService.request( + user: @user, + email: 'replacement@example.org' + ) + + assert_equal first.id, replacement.id + assert_equal 'replacement@example.org', replacement.email + assert_nil AdditionalNotificationEmail.record_for_token(old_token) + assert_equal 1, + @user.additional_notification_email_audits.where(event: 'address_replaced').count + end + + def test_resend_invalidates_the_previous_link_and_is_rate_limited + record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + old_token = record.verification_token + + AdditionalNotificationEmailService.resend(user: @user) + AdditionalNotificationEmailService.resend(user: @user) + + assert_nil AdditionalNotificationEmail.record_for_token(old_token) + assert_raises(AdditionalNotificationEmailService::RateLimited) do + AdditionalNotificationEmailService.resend(user: @user) + end + assert_equal 3, + @user.additional_notification_email_audits.where(event: 'verification_requested').count + end + + def test_verification_locks_the_user_before_touching_the_address + record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + token = record.verification_token + locks = [] + user = record.user + user.define_singleton_method(:with_lock) do |*args, &block| + locks << :user + super(*args, &block) + end + record.define_singleton_method(:with_lock) do |*args, &block| + locks << :address + super(*args, &block) + end + + AdditionalNotificationEmail.stub(:record_for_token, ->(_token) { locks.include?(:user) ? record.reload : record }) do + AdditionalNotificationEmailService.verify(token: token) + end + + # request, resend and remove lock the user first. The same order here + # cannot deadlock against them. + assert_equal [:user], locks + assert record.reload.verified? + end + + def test_removal_stops_future_use_and_keeps_a_token_free_audit + record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + token = record.verification_token + + assert AdditionalNotificationEmailService.remove(user: @user) + assert_nil @user.reload.additional_notification_email + assert_nil AdditionalNotificationEmail.record_for_token(token) + + audit = @user.additional_notification_email_audits.find_by!(event: 'removed') + assert_equal %w[id user_id event created_at updated_at].sort, audit.attributes.keys.sort + end +end diff --git a/test/models/notification_discussion_request_test.rb b/test/models/notification_discussion_request_test.rb new file mode 100644 index 0000000000..7a9c24b27c --- /dev/null +++ b/test/models/notification_discussion_request_test.rb @@ -0,0 +1,161 @@ +require 'test_helper' +require 'minitest/mock' +require 'tempfile' + +# EN-V08: OnTrack has no discussion booking record. This proposal notifies the +# student when a tutor raises the audio discussion request that exists today. +class NotificationDiscussionRequestTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @tutor = @project.tutor_for(@task_definition) + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + mail.multipart? ? mail.parts.map { |part| part.body.decoded }.join("\n") : mail.body.decoded + end + + def build_wav_upload + sample_rate = 8_000 + samples = Array.new(800, 0).pack('s<*') + header = [ + 'RIFF', 36 + samples.bytesize, 'WAVE', + 'fmt ', 16, 1, 1, sample_rate, sample_rate * 2, 2, 16, + 'data', samples.bytesize + ].pack('A4VA4A4VvvVVvvA4V') + + tempfile = Tempfile.new(['discussion-request', '.wav']) + tempfile.binmode + tempfile.write(header) + tempfile.write(samples) + tempfile.rewind + + { + 'filename' => 'discussion-request.wav', + 'type' => 'audio/wav', + 'tempfile' => tempfile + } + end + + def with_audio_uploads(count) + uploads = Array.new(count) { build_wav_upload } + yield uploads + ensure + uploads&.each do |upload| + upload['tempfile'].close! + rescue Errno::ENOENT + upload['tempfile'].close + end + end + + def notification_target + DiscussionComment.new(recipient: @student) + end + + def test_multiple_audio_prompts_create_one_notification_after_upload + discussion = nil + + with_audio_uploads(2) do |uploads| + assert_difference 'Notification.count', 1 do + discussion = @task.add_discussion_comment(@tutor, uploads) + end + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert discussion.persisted? + assert_equal 2, discussion.number_of_prompts + assert_equal @student, notification.user + assert_equal 'feedback', notification.notification_type + assert_equal 'discussion_request_created', notification.event + assert_equal 'A discussion prompt is ready for you.', notification.message + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}/feedback" + ) + + # Email is queued rather than sent inline since EN-F03. + NotificationEmailJob.drain + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + def test_feedback_preference_suppresses_every_channel + @student.update!(receive_feedback_notifications: false) + + assert_no_difference 'Notification.count' do + @task.send(:notify_discussion_request_recipient, notification_target) + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_email_uses_the_event_template_without_assessment_content + private_task_name = 'PRIVATE-TASK-NAME-7429' + private_unit_name = 'PRIVATE-UNIT-NAME-7429' + private_tutor_name = 'PrivateTutor7429 SensitiveName7429' + + @task_definition.update!(name: private_task_name) + @unit.update!(name: private_unit_name) + @tutor.update!(first_name: 'PrivateTutor7429', last_name: 'SensitiveName7429') + + @task.send(:notify_discussion_request_recipient, notification_target) + NotificationEmailJob.drain + + body = delivered_body + + assert_not_empty body, 'guard: the multipart email body must be readable' + assert_includes body, 'The prompt is not included in this email' + assert_includes body, 'feedback notifications are turned on' + assert_not_includes body, private_task_name + assert_not_includes body, private_unit_name + assert_not_includes body, private_tutor_name + end + + def test_failed_audio_attachment_does_not_send_a_notification + invalid = Tempfile.new(['invalid-discussion-request', '.wav']) + invalid.write('not audio') + invalid.rewind + upload = { + 'filename' => 'invalid-discussion-request.wav', + 'type' => 'audio/wav', + 'tempfile' => invalid + } + + assert_no_difference 'Notification.count' do + assert_raises RuntimeError do + @task.add_discussion_comment(@tutor, [upload]) + end + end + + assert_empty ActionMailer::Base.deliveries + ensure + invalid&.close! + end + + def test_notification_failure_does_not_stop_the_request_being_created + discussion = nil + + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification failed' } do + with_audio_uploads(1) do |uploads| + discussion = @task.add_discussion_comment(@tutor, uploads) + end + end + + assert_not_nil discussion + assert discussion.persisted? + assert_equal @student, discussion.recipient + end +end diff --git a/test/models/notification_extension_request_test.rb b/test/models/notification_extension_request_test.rb new file mode 100644 index 0000000000..f055514c88 --- /dev/null +++ b/test/models/notification_extension_request_test.rb @@ -0,0 +1,209 @@ +require 'test_helper' +require 'cgi' +require 'minitest/mock' + +# A student's extension request notifies whoever has to assess it, and only +# while it is still waiting on a person. +class NotificationExtensionRequestTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + EXTENSION_REQUEST_TEXT = + 'Private reason for the extension that must stay inside OnTrack.'.freeze + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + # Leave room for a one week extension before the due date. + @task_definition.update!(due_date: @task_definition.target_date + 2.weeks) + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @convenor = @unit.main_convenor_user + + # Leave the request for a person to assess unless a test says otherwise. + @unit.update!(auto_apply_extension_before_deadline: false) + end + + # Without a tutorial, tutor_for falls back to the main convenor, so give the + # student a tutor of their own when a test needs the two to differ. + def give_the_student_a_tutor + tutor_role = @unit.employ_staff(FactoryBot.create(:user, :tutor), Role.tutor) + @project.enrol_in(FactoryBot.create(:tutorial, unit: @unit, campus: @project.campus, unit_role: tutor_role)) + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + tutor_role.user + end + + def request_extension(by_user: @student) + @task.apply_for_extension(by_user, EXTENSION_REQUEST_TEXT, 1) + end + + def extension_requests + Notification.where(event: 'extension_requested') + end + + def delivered_parts + mail = ActionMailer::Base.deliveries.last + + { + html: mail&.html_part&.body&.decoded.to_s, + text: mail&.text_part&.body&.decoded.to_s + } + end + + def test_a_student_request_notifies_their_tutor + tutor = give_the_student_a_tutor + extension = nil + + assert_difference 'Notification.count', 1 do + extension = request_extension + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + product_name = Doubtfire::Application.config.institution[:product_name] + expected_message = "#{@student.name} asked for an extension on #{@task_definition.name} in #{product_name}." + + assert_not extension.assessed?, 'guard: the request must still be waiting on a person' + assert_equal tutor, notification.user + assert_equal 'task', notification.notification_type + assert_equal 'extension_requested', notification.event + assert_equal expected_message, notification.message + assert_equal extension, notification.notifiable + + push = assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}", + expected_body: 'A student asked for an extension.' + ) + assert_not_includes push['body'], EXTENSION_REQUEST_TEXT + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [tutor.email], ActionMailer::Base.deliveries.last.to + + parts = delivered_parts + + parts.each_value do |body| + assert_not_empty body + assert_includes body, "Hi #{tutor.first_name}" + assert_includes body, notification.web_path + assert_includes body, '/edit_profile' + assert_not_includes body, EXTENSION_REQUEST_TEXT + end + + assert_includes parts[:text], expected_message + assert_includes parts[:html], CGI.escapeHTML(expected_message) + end + + def test_with_no_tutor_the_request_goes_to_the_main_convenor + assert_equal @convenor, @project.tutor_for(@task_definition), 'guard: the student must have no tutor' + + assert_difference 'Notification.count', 1 do + request_extension + end + + notification = Notification.recent_first.first + + assert_equal 'extension_requested', notification.event + assert_equal @convenor, notification.user + end + + # The request is a task notification so the tutor's "Task notifications" + # switch governs it. Turned off, nothing is created, emailed or pushed. + def test_a_tutor_with_task_notifications_off_gets_no_email_or_push + tutor = give_the_student_a_tutor + tutor.update!(receive_task_notifications: false) + PushNotificationDeliveryJob.clear + + assert_no_difference 'Notification.count' do + request_extension + end + NotificationEmailJob.drain + + assert_empty ActionMailer::Base.deliveries + assert_empty PushNotificationDeliveryJob.jobs + end + + # The convenor is not the student's tutor here, so the rule against notifying + # yourself is not what stops it. The extension is assessed on the spot; the + # next test covers the student-only check for when that assessment fails. + def test_an_extension_created_by_staff_does_not_notify_the_tutor + tutor = give_the_student_a_tutor + extension = nil + + assert_no_difference -> { extension_requests.count } do + extension = request_extension(by_user: @convenor) + end + + assert extension.assessed?, 'guard: staff extensions are assessed as they are made' + assert_empty tutor.notifications + end + + # A staff extension that cannot be applied is left unassessed, and it still + # must not reach the tutor as a request from the student. + def test_a_staff_extension_that_cannot_be_applied_does_not_notify_the_tutor + tutor = give_the_student_a_tutor + extension = nil + + @task.stub :grant_extension, false do + assert_no_difference -> { extension_requests.count } do + extension = request_extension(by_user: @convenor) + end + end + + assert_not extension.assessed?, 'guard: the failed grant must leave it unassessed' + assert_empty tutor.notifications + end + + def test_an_automatically_approved_request_does_not_notify_the_tutor + @unit.update!(auto_apply_extension_before_deadline: true) + tutor = give_the_student_a_tutor + extension = nil + + assert_no_difference -> { extension_requests.count } do + extension = request_extension + end + + assert extension.assessed?, 'guard: the unit must have approved it already' + assert extension.extension_granted + assert_empty tutor.notifications + end + + # If the automatic approval cannot be applied the request is left waiting, + # so somebody still has to hear about it. + def test_a_failed_automatic_approval_still_notifies_the_tutor + @unit.update!(auto_apply_extension_before_deadline: true) + tutor = give_the_student_a_tutor + extension = nil + + @task.stub :grant_extension, false do + assert_difference -> { extension_requests.count }, 1 do + extension = request_extension + end + end + + assert_not extension.assessed? + assert_equal tutor, extension_requests.last.user + end + + def test_a_notification_failure_does_not_lose_the_request + extension = nil + calls = 0 + exploding_notify = lambda do |**_kwargs| + calls += 1 + raise StandardError, 'notification exploded' + end + + NotificationService.stub :notify, exploding_notify do + extension = request_extension + end + + assert_equal 1, calls, 'guard: the notification has to have been attempted' + assert extension.persisted?, 'the extension request must still be saved' + assert_not extension.assessed? + end +end diff --git a/test/models/notification_extension_test.rb b/test/models/notification_extension_test.rb new file mode 100644 index 0000000000..1ec5a8e0ba --- /dev/null +++ b/test/models/notification_extension_test.rb @@ -0,0 +1,191 @@ +require 'test_helper' +require 'minitest/mock' + +# EN-E03: assessing an extension request notifies the student. +class NotificationExtensionTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + EXTENSION_REQUEST_TEXT = + 'Private extension request text that must stay inside OnTrack.'.freeze + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task_definition.update!(due_date: @task_definition.target_date + 2.weeks) + @task = @project.task_for_task_definition(@task_definition) + + @student = @project.student + @tutor = @project.tutor_for(@task_definition) + + # Prevent the request being assessed automatically when it is created. + @unit.update!(auto_apply_extension_before_deadline: false) + end + + def create_extension_request + extension = @task.apply_for_extension( + @student, + EXTENSION_REQUEST_TEXT, + 1 + ) + + # The request itself emails the tutor (extension_requested). Drop that job + # so these tests only count what the assessment sends the student. + NotificationEmailJob.clear + + extension + end + + def delivered_parts + mail = ActionMailer::Base.deliveries.last + + { + html: mail&.html_part&.body&.decoded.to_s, + text: mail&.text_part&.body&.decoded.to_s + } + end + + def test_granted_extension_notifies_student_with_new_date + extension = create_extension_request + + assert_difference 'Notification.count', 1 do + extension.assess_extension(@tutor, true) + end + NotificationEmailJob.drain + + extension.reload + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_equal 'extension', notification.notification_type + assert_equal 'extension_assessed', notification.event + push = assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}", + # The lock-screen body is the reviewed override, not the raw message: the + # raw message names the outcome and a date, both banned on a lock screen. + expected_body: 'An extension request was assessed.' + ) + assert_not_includes notification.message, EXTENSION_REQUEST_TEXT + assert_not_includes push['body'], EXTENSION_REQUEST_TEXT + + assert extension.extension_granted + assert_includes notification.message, 'Extension granted' + assert_includes( + notification.message, + @task.reload.due_date.strftime('%a %b %e') + ) + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + + parts = delivered_parts + + assert_not_empty parts[:html] + assert_not_empty parts[:text] + + assert_includes parts[:html], notification.message + assert_includes parts[:text], notification.message + + assert_includes parts[:html], notification.link + assert_includes parts[:text], notification.link + end + + def test_denied_extension_notifies_student + extension = create_extension_request + + assert_difference 'Notification.count', 1 do + extension.assess_extension(@tutor, false) + end + NotificationEmailJob.drain + + extension.reload + notification = Notification.recent_first.first + + assert_not extension.extension_granted + + assert_equal @student, notification.user + assert_equal 'extension', notification.notification_type + assert_equal 'extension_assessed', notification.event + assert_equal 'Extension rejected', notification.message + + assert_equal 1, ActionMailer::Base.deliveries.count + + parts = delivered_parts + + assert_includes parts[:html], 'Extension rejected' + assert_includes parts[:text], 'Extension rejected' + end + + def test_already_assessed_extension_does_not_send_another_notification + extension = create_extension_request + + extension.assess_extension(@tutor, false) + + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + assert_no_difference 'Notification.count' do + result = extension.assess_extension(@tutor, true) + + assert_equal false, result + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_deadline_error_does_not_send_notification + extension = create_extension_request + + ActionMailer::Base.deliveries.clear + + @task.stub :can_apply_for_extension?, false do + assert_no_difference 'Notification.count' do + extension.assess_extension(@tutor, true) + end + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_failed_grant_does_not_assess_or_notify_student + extension = create_extension_request + task = extension.task + original_extensions = task.extensions + + task.stub :can_apply_for_extension?, true do + task.stub :grant_extension, false do + assert_no_difference 'Notification.count' do + result = extension.assess_extension(@tutor, true) + + assert_equal false, result + end + end + end + + assert_empty ActionMailer::Base.deliveries + assert_includes extension.errors[:extension], 'could not be applied' + assert_not extension.assessed? + assert_not extension.extension_granted + assert_equal original_extensions, task.reload.extensions + + extension.reload + assert_not extension.assessed? + assert_not extension.extension_granted + end + + def test_extension_notification_uses_event_specific_templates + extension = create_extension_request + + extension.assess_extension(@tutor, false) + NotificationEmailJob.drain + + parts = delivered_parts + + assert_includes parts[:html], 'Your extension request has been assessed' + assert_includes parts[:text], 'Your extension request has been assessed' + end +end diff --git a/test/models/notification_group_test.rb b/test/models/notification_group_test.rb new file mode 100644 index 0000000000..5b02e14a47 --- /dev/null +++ b/test/models/notification_group_test.rb @@ -0,0 +1,177 @@ +require 'test_helper' +require 'minitest/mock' +require 'tempfile' + +# EN-V05: notify only the affected student when group membership changes. +class NotificationGroupTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @group = FactoryBot.create(:group, unit: @project.unit) + @student = @project.student + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + mail.multipart? ? mail.parts.map { |part| part.body.decoded }.join("\n") : mail.body.decoded + end + + def test_adding_a_member_notifies_only_that_student + assert_difference 'Notification.count', 1 do + @group.add_member(@project) + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_equal 'general', notification.notification_type + assert_equal 'group_membership_changed', notification.event + assert_equal( + "You have been added to group #{@group.name} in #{@project.unit.code}.", + notification.message + ) + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/groups", + expected_body: 'Your group membership changed.' + ) + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + + # Confirms the event-specific template is being used. + assert_includes delivered_body, 'group membership changed' + assert_includes delivered_body, notification.message + end + + def test_removing_a_member_notifies_that_student + @group.add_member(@project) + + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + assert_difference 'Notification.count', 1 do + @group.remove_member(@project) + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_equal 'general', notification.notification_type + assert_equal 'group_membership_changed', notification.event + assert_includes notification.message, 'removed from' + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + def test_other_group_members_are_not_notified + other_project = FactoryBot.create(:project, unit: @project.unit) + + @group.add_member(other_project) + + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + assert_difference 'Notification.count', 1 do + @group.add_member(@project) + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_not_equal other_project.student, notification.user + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + def test_switch_to_tutorial_sends_tutorial_changes_without_leave_then_join_notifications + unit = FactoryBot.create( + :unit, + group_sets: 1, + groups: [{ gs: 0, students: 0 }] + ) + + group_set = unit.group_sets.first + group_set.update!( + keep_groups_in_same_class: true, + allow_students_to_manage_groups: true + ) + + group = group_set.groups.first + + project_one = group.tutorial.projects.first + project_two = group.tutorial.projects.last + + group.add_member(project_one) + group.add_member(project_two) + + new_tutorial = FactoryBot.create(:tutorial, unit: unit, campus: nil) + + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + assert_difference -> { Notification.where(event: 'tutorial_changed').count }, 2 do + assert_no_difference -> { Notification.where(event: 'group_membership_changed').count } do + group.switch_to_tutorial(new_tutorial) + end + end + NotificationEmailJob.drain + + tutorial_notifications = Notification.where(event: 'tutorial_changed').recent_first.limit(2) + + assert_equal( + [project_one.student.id, project_two.student.id].sort, + tutorial_notifications.map(&:user_id).sort + ) + assert_equal 2, ActionMailer::Base.deliveries.count + assert_equal( + [project_one.student.email, project_two.student.email].sort, + ActionMailer::Base.deliveries.flat_map(&:to).sort + ) + end + + def test_notification_failure_does_not_stop_membership_change + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification failed' } do + assert_nothing_raised do + @group.add_member(@project) + end + end + + assert_includes @group.reload.projects, @project + end + + def test_bulk_csv_import_adds_member_without_notification + Tempfile.create(['student-groups', '.csv']) do |file| + file.write("group_name,username\n#{@group.name},#{@student.username}\n") + file.flush + + notification_calls = 0 + + NotificationService.stub :notify, ->(**_kwargs) { notification_calls += 1 } do + result = @project.unit.import_student_groups_from_csv( + @group.group_set, + file.path + ) + + assert_empty result[:errors], result.inspect + assert_empty result[:ignored], result.inspect + assert_equal 1, result[:success].count, result.inspect + end + + assert_equal 0, notification_calls + end + + assert_includes @group.reload.projects, @project + end +end diff --git a/test/models/notification_new_task_test.rb b/test/models/notification_new_task_test.rb new file mode 100644 index 0000000000..88541d86cd --- /dev/null +++ b/test/models/notification_new_task_test.rb @@ -0,0 +1,287 @@ +# frozen_string_literal: true + +require 'test_helper' +require 'minitest/mock' + +# EN-V02: newly available tasks notify eligible students. +class NotificationNewTaskTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @unit = FactoryBot.create( + :unit, + with_students: false, + task_count: 0, + tutorials: 0, + outcome_count: 0, + staff_count: 0, + campus_count: 1, + active: true, + start_date: Time.zone.now - 1.week, + end_date: Time.zone.now + 12.weeks + ) + + @campus = Campus.first + + @student = FactoryBot.create( + :user, + :student, + receive_task_notifications: true + ) + + @project = FactoryBot.create( + :project, + unit: @unit, + campus: @campus, + user: @student, + enrolled: true, + target_grade: 2 + ) + + @task_definition = FactoryBot.create( + :task_definition, + unit: @unit, + outcome_count: 0, + target_grade: 1, + start_date: Time.zone.now - 1.day, + target_date: Time.zone.now + 1.week, + due_date: Time.zone.now + 2.weeks + ) + end + + def run_job + NewTaskAvailableNotificationJob.new.perform(@task_definition.id) + NotificationEmailJob.drain + end + + def event_notifications + Notification.where(event: 'new_task_available') + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + if mail.multipart? + mail.parts.map { |part| part.body.decoded }.join("\n") + else + mail.body.decoded + end + end + + def test_available_task_notifies_eligible_student + assert_difference 'Notification.count', 1 do + assert_no_difference 'Task.count' do + run_job + end + end + + notification = event_notifications.last + + assert_equal @student, notification.user + assert_equal 'task', notification.notification_type + assert_equal 'new_task_available', notification.event + + assert_equal( + "A new task is available: #{@task_definition.abbreviation} in #{@unit.code}.", + notification.message + ) + + assert_equal( + "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}", + notification.link + ) + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}" + ) + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + + body = delivered_body + + assert_not_empty body + assert_includes body, 'A new task is now available' + assert_includes body, @task_definition.abbreviation + end + + def test_fans_out_to_each_eligible_student + second_student = FactoryBot.create( + :user, + :student, + receive_task_notifications: true + ) + + FactoryBot.create( + :project, + unit: @unit, + campus: @campus, + user: second_student, + enrolled: true, + target_grade: 2 + ) + + assert_difference 'Notification.count', 2 do + run_job + end + + recipients = event_notifications.includes(:user).map(&:user) + + assert_includes recipients, @student + assert_includes recipients, second_student + assert_equal 2, ActionMailer::Base.deliveries.count + end + + def test_student_with_task_notifications_disabled_is_not_notified + @student.update!(receive_task_notifications: false) + + assert_no_difference 'Notification.count' do + run_job + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_unenrolled_student_is_not_notified + @project.update!(enrolled: false) + + assert_no_difference 'Notification.count' do + run_job + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_student_below_task_target_grade_is_not_notified + @project.update!(target_grade: 0) + + assert_no_difference 'Notification.count' do + run_job + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_inactive_unit_does_not_send_notifications + @unit.update!(active: false) + + assert_no_difference 'Notification.count' do + run_job + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_future_effective_student_start_date_is_not_notified + @unit.update!(allow_flexible_dates: true) + + task = @project.task_for_task_definition(@task_definition) + + task.update!( + target_start_date: Time.zone.now + 2.days + ) + + assert_operator task.local_start_date.to_date, :>, Time.zone.today + + assert_no_difference 'Notification.count' do + run_job + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_notification_failure_makes_job_fail_for_retry + notification_failure = lambda do |_notification| + raise StandardError, 'temporary notification failure' + end + + NotificationService.stub(:deliver, notification_failure) do + error = assert_raises(RuntimeError) do + run_job + end + + assert_includes error.message, @project.id.to_s + end + + notification = event_notifications.find_by!(user: @student) + assert_nil notification.delivered_at + + assert_no_difference 'Notification.count' do + run_job + end + + assert_not_nil notification.reload.delivered_at + assert_equal 1, ActionMailer::Base.deliveries.count + end + + def test_job_has_limited_retries + assert_equal( + 3, + NewTaskAvailableNotificationJob.get_sidekiq_options['retry'] + ) + end + + def test_running_fan_out_twice_does_not_duplicate_notification + run_job + + assert_equal 1, event_notifications.count + assert_equal 1, ActionMailer::Base.deliveries.count + + assert_no_difference 'Notification.count' do + run_job + end + + assert_equal 1, event_notifications.count + assert_equal 1, ActionMailer::Base.deliveries.count + end + + def test_delivery_rechecks_a_stale_project_after_withdrawal + stale_project = Project.find(@project.id) + @project.update!(enrolled: false) + + assert_no_difference 'Notification.count' do + NewTaskAvailableNotificationJob.deliver(stale_project, @task_definition) + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_renaming_a_task_does_not_send_a_second_availability_notification + run_job + @task_definition.update!(abbreviation: "RENAMED#{SecureRandom.hex(3)}") + + assert_no_difference 'Notification.count' do + run_job + end + + assert_equal 1, ActionMailer::Base.deliveries.count + end + + def test_reusing_an_abbreviation_for_a_new_task_still_notifies + reused_abbreviation = @task_definition.abbreviation + run_job + @task_definition.update!(abbreviation: "RETIRED#{SecureRandom.hex(3)}") + + replacement = FactoryBot.create( + :task_definition, + unit: @unit, + outcome_count: 0, + abbreviation: reused_abbreviation, + target_grade: 1, + start_date: 1.day.ago, + target_date: 1.week.from_now + ) + + assert_difference 'Notification.count', 1 do + NewTaskAvailableNotificationJob.new.perform(replacement.id) + end + # This call bypasses run_job, so it has to drain the mail queue itself. + NotificationEmailJob.drain + + assert_equal 2, ActionMailer::Base.deliveries.count + end +end diff --git a/test/models/notification_portfolio_submitted_test.rb b/test/models/notification_portfolio_submitted_test.rb new file mode 100644 index 0000000000..c8f2d29475 --- /dev/null +++ b/test/models/notification_portfolio_submitted_test.rb @@ -0,0 +1,115 @@ +require 'test_helper' + +# A student's new portfolio submission tells the staff who will mark it. +class NotificationPortfolioSubmittedTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + include TestHelpers::PushNotificationHelper + + def app + Rails.application + end + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @student = @project.student + + @project.tutorial_enrolments.destroy_all + tutor_role = @unit.employ_staff(FactoryBot.create(:user, :tutor), Role.tutor) + @project.enrol_in(FactoryBot.create(:tutorial, unit: @unit, campus: @project.campus, unit_role: tutor_role)) + @tutor = tutor_role.user + @project.reload + end + + def submit_portfolio(as: @student, value: true) + add_auth_header_for(user: as) + put_json("/api/projects/#{@project.id}", id: @project.id, compile_portfolio: value) + + assert_equal 200, last_response.status, last_response.body + end + + def staff_notifications + Notification.where(event: 'portfolio_submitted') + end + + def test_the_students_tutor_is_told_once + assert_difference -> { staff_notifications.count }, 1 do + submit_portfolio + end + + notification = staff_notifications.first + + assert_equal @tutor, notification.user + assert_equal 'portfolio', notification.notification_type + assert_equal "#{@student.name} submitted a portfolio in #{@unit.code}.", notification.message + assert_equal "/projects/#{@project.id}/dashboard", notification.link + assert_equal @project, notification.notifiable + assert_equal @student.id, notification.target_ids[:student_id] + assert_equal @unit.id, notification.target_ids[:unit_id] + + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard", + expected_body: 'A student submitted a portfolio.' + ) + end + + def test_the_student_still_gets_their_own_receipt + assert_difference -> { Notification.where(user: @student, event: 'portfolio_received').count }, 1 do + submit_portfolio + end + end + + def test_a_student_with_no_tutor_tells_the_main_convenor + @project.tutorial_enrolments.destroy_all + @project.reload + + submit_portfolio + + assert_equal [@unit.main_convenor_user], staff_notifications.map(&:user) + end + + def test_the_tutors_portfolio_preference_switches_it_off + @tutor.update!(receive_portfolio_notifications: false) + + assert_no_difference -> { staff_notifications.count } do + submit_portfolio + end + end + + def test_repeating_the_same_submission_does_not_tell_the_tutor_again + submit_portfolio + + assert_no_difference -> { staff_notifications.count } do + submit_portfolio + end + end + + def test_staff_submitting_on_the_students_behalf_are_not_told_about_their_own_action + assert_no_difference -> { staff_notifications.where(user: @tutor).count } do + submit_portfolio(as: @tutor) + end + end + + def test_one_email_for_the_tutor + submit_portfolio + NotificationEmailJob.drain + + tutor_mail = ActionMailer::Base.deliveries.select { |mail| mail.to.include?(@tutor.email) } + staff_url = "/units/#{@unit.id}/students/portfolios/#{@project.id}" + + assert_equal 1, tutor_mail.count + [tutor_mail.first.html_part.body.decoded, tutor_mail.first.text_part.body.decoded].each do |body| + assert_includes body, "Hi #{@tutor.first_name}" + assert_includes body, "#{@student.name} submitted a portfolio" + assert_includes body, staff_url + assert_includes body, '/edit_profile' + end + assert_includes tutor_mail.first.html_part.body.decoded, 'Portfolio submitted' + end +end diff --git a/test/models/notification_portfolio_test.rb b/test/models/notification_portfolio_test.rb new file mode 100644 index 0000000000..3cd8ec73de --- /dev/null +++ b/test/models/notification_portfolio_test.rb @@ -0,0 +1,200 @@ +require 'test_helper' +require 'minitest/mock' + +# EN-V07: a newly accepted manual portfolio submission sends one receipt to +# the submitting student. +class NotificationPortfolioTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + include TestHelpers::PushNotificationHelper + + def app + Rails.application + end + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @project.campus.update!(timezone: 'Australia/Melbourne') + @student = @project.student + + # The same submission also tells the student's tutors. This file is about + # the student's receipt, so their staff copy is switched off here and + # covered on its own in notification_portfolio_submitted_test.rb. + staff = @project.tutorial_enrolments.filter_map { |enrolment| enrolment.tutorial&.tutor } + (staff + [@project.main_convenor_user]).compact.uniq.each do |user| + user.update!(receive_portfolio_notifications: false) + end + + add_auth_header_for(user: @student) + end + + def submit_portfolio(value: true) + # Some receipt assertions freeze time. Mint the request token inside that + # clock so the authentication expiry is evaluated against the same instant. + add_auth_header_for(user: @student) + + put_json( + "/api/projects/#{@project.id}", + id: @project.id, + compile_portfolio: value + ) + + assert_equal 200, last_response.status, last_response.body + + # Email is queued rather than sent inline since EN-F03. Draining here keeps + # every deliveries assertion in this file reading the way it did before, + # and it is the same shape as run_job in the other notification tests. + NotificationEmailJob.drain + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + mail.multipart? ? mail.parts.map { |part| part.body.decoded }.join("\n") : mail.body.decoded + end + + def test_a_new_portfolio_submission_sends_one_receipt_to_the_student + travel_to Time.zone.parse('2026-08-23 12:34:00 UTC') do + assert_difference 'Notification.count', 1 do + submit_portfolio + end + end + NotificationEmailJob.drain + + notification = Notification.find_by!(user: @student, event: 'portfolio_received') + + assert_equal @student, notification.user + assert_equal 'portfolio', notification.notification_type + assert_equal 'portfolio_received', notification.event + assert_equal( + "#{Doubtfire::Application.config.institution[:product_name]} received your " \ + 'portfolio submission at 23 August 2026 at 10:34 PM AEST (UTC+10:00).', + notification.message + ) + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard", + expected_body: 'Your portfolio submission was received.' + ) + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + assert_equal( + "#{Doubtfire::Application.config.institution[:product_name]}: Portfolio received", + ActionMailer::Base.deliveries.last.subject + ) + end + + def test_the_receipt_uses_the_event_template_and_excludes_assessment_content + assessment_content = 'PRIVATE-ASSESSMENT-RATIONALE-7429' + @project.update!(grade_rationale: assessment_content, submitted_grade: 3) + + travel_to Time.zone.parse('2026-08-23 12:34:00 UTC') do + submit_portfolio + end + NotificationEmailJob.drain + + body = delivered_body + + assert_not_empty body, 'guard: the body must be readable or the privacy assertions prove nothing' + assert_includes body, '23 August 2026 at 10:34 PM AEST (UTC+10:00)' + assert_includes body, 'This receipt confirms when the submission was received' + assert_includes body, 'It does not confirm an assessment outcome' + assert_not_includes body, assessment_content + end + + def test_portfolio_preference_suppresses_every_notification_channel + @student.update!(receive_portfolio_notifications: false) + + assert_no_difference 'Notification.count' do + submit_portfolio + end + + assert_empty ActionMailer::Base.deliveries + assert @project.reload.compile_portfolio? + assert_not_nil @project.portfolio_submission_date + end + + def test_retrying_a_pending_manual_submission_does_not_send_a_second_receipt + travel_to Time.zone.parse('2026-08-23 12:34:00 UTC') do + submit_portfolio + end + + original_submission_date = @project.reload.portfolio_submission_date + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + travel_to Time.zone.parse('2026-08-23 12:39:00 UTC') do + assert_no_difference 'Notification.count' do + submit_portfolio + end + end + + assert_empty ActionMailer::Base.deliveries + assert_equal original_submission_date, @project.reload.portfolio_submission_date + end + + def test_a_later_resubmission_receives_a_new_receipt + travel_to Time.zone.parse('2026-08-23 12:34:00 UTC') do + submit_portfolio + end + NotificationEmailJob.drain + + first_submission_date = @project.reload.portfolio_submission_date + @project.update!(compile_portfolio: false) + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + travel_to Time.zone.parse('2026-08-24 01:15:00 UTC') do + assert_difference 'Notification.count', 1 do + submit_portfolio + end + end + NotificationEmailJob.drain + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_operator @project.reload.portfolio_submission_date, :>, first_submission_date + end + + def test_a_manual_submission_replaces_a_pending_auto_generated_portfolio + @project.update!( + compile_portfolio: true, + portfolio_auto_generated: true, + portfolio_submission_date: nil + ) + + assert_difference 'Notification.count', 1 do + submit_portfolio + end + NotificationEmailJob.drain + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_not @project.reload.portfolio_auto_generated? + assert_not_nil @project.portfolio_submission_date + end + + def test_cancelling_portfolio_generation_does_not_send_a_receipt + assert_no_difference 'Notification.count' do + submit_portfolio(value: false) + end + + assert_empty ActionMailer::Base.deliveries + assert_nil @project.reload.portfolio_submission_date + end + + def test_a_notification_failure_does_not_reject_the_submission + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification failed' } do + assert_nothing_raised do + submit_portfolio + end + end + + assert @project.reload.compile_portfolio? + assert_not_nil @project.portfolio_submission_date + end +end diff --git a/test/models/notification_target_ids_test.rb b/test/models/notification_target_ids_test.rb new file mode 100644 index 0000000000..af5ab296e1 --- /dev/null +++ b/test/models/notification_target_ids_test.rb @@ -0,0 +1,195 @@ +require 'test_helper' + +# The ids a client needs to open the page a notification is about. Worked out +# from the notifiable first and the link second, and nil once a record is gone. +class NotificationTargetIdsTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + + def app + Rails.application + end + + setup do + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @tutor = @project.tutor_for(@task_definition) + end + + def task_link(project = @project, abbreviation = @task_definition.abbreviation) + "/projects/#{project.id}/dashboard/#{ERB::Util.url_encode(abbreviation)}" + end + + def notification_for(notifiable: nil, link: nil, user: @student) + Notification.create!( + user: user, + notification_type: 'task', + event: 'test_event', + message: 'Something happened.', + link: link, + notifiable: notifiable + ) + end + + def assert_task_ids(ids) + assert_equal @unit.id, ids[:unit_id] + assert_equal @project.id, ids[:project_id] + assert_equal @student.id, ids[:student_id] + assert_equal @task_definition.id, ids[:task_definition_id] + assert_equal @task_definition.abbreviation, ids[:task_definition_abbr] + end + + def test_a_comment_notification_names_the_comment_and_its_task + comment = @task.add_text_comment(@student, 'A question for my tutor') + ids = Notification.find_by!(notifiable: comment).target_ids + + assert_task_ids(ids) + assert_equal @task.id, ids[:task_id] + assert_equal comment.id, ids[:comment_id] + assert_nil ids[:group_id] + end + + def test_a_task_notification_names_the_task + ids = notification_for(notifiable: @task, link: task_link).target_ids + + assert_task_ids(ids) + assert_equal @task.id, ids[:task_id] + assert_nil ids[:comment_id] + end + + def test_a_link_only_notification_is_resolved_from_its_link + ids = notification_for(link: task_link).target_ids + + assert_task_ids(ids) + assert_equal @task.id, ids[:task_id] + end + + def test_a_task_link_with_no_task_row_yet_still_names_the_definition + other_definition = @unit.task_definitions.second + Task.where(project: @project, task_definition: other_definition).delete_all + + ids = notification_for(link: task_link(@project, other_definition.abbreviation)).target_ids + + assert_equal other_definition.id, ids[:task_definition_id] + assert_nil ids[:task_id] + end + + def test_a_project_link_names_the_project_and_unit_only + ids = notification_for(link: "/projects/#{@project.id}/dashboard").target_ids + + assert_equal @unit.id, ids[:unit_id] + assert_equal @project.id, ids[:project_id] + assert_equal @student.id, ids[:student_id] + assert_nil ids[:task_definition_id] + assert_nil ids[:task_id] + end + + def test_a_portfolio_notification_on_the_project_names_the_student + ids = notification_for(notifiable: @project, link: "/projects/#{@project.id}/dashboard", user: @tutor).target_ids + + assert_equal @project.id, ids[:project_id] + assert_equal @student.id, ids[:student_id] + end + + def test_a_group_change_names_the_group_and_the_project + group_set = FactoryBot.create(:group_set, unit: @unit) + group = FactoryBot.create(:group, group_set: group_set) + + ids = notification_for(notifiable: group, link: "/projects/#{@project.id}/groups").target_ids + + assert_equal group.id, ids[:group_id] + assert_equal @project.id, ids[:project_id] + end + + def test_a_missing_project_leaves_every_id_nil + ids = notification_for(link: '/projects/999999999/dashboard/1.1P').target_ids + + assert(ids.values.all?(&:nil?), ids.inspect) + end + + def test_an_unknown_task_abbreviation_leaves_the_task_ids_nil + ids = notification_for(link: task_link(@project, 'NOPE')).target_ids + + assert_equal @project.id, ids[:project_id] + assert_nil ids[:task_definition_id] + assert_nil ids[:task_definition_abbr] + assert_nil ids[:task_id] + end + + def test_a_general_notification_with_no_link_has_no_target + ids = notification_for.target_ids + + assert(ids.values.all?(&:nil?), ids.inspect) + end + + # web_path mirrors the web client's notification-target.ts, so an email + # button opens the page the bell would. + def test_web_path_for_a_student_opens_their_own_pages + comment_link = "#{task_link}/feedback" + project = "/projects/#{@project.id}" + abbreviation = ERB::Util.url_encode(@task_definition.abbreviation) + + { + ['task_comment_created', 'feedback', comment_link] => "#{project}/dashboard/#{abbreviation}/feedback", + ['extension_assessed', 'extension', task_link] => "#{project}/dashboard/#{abbreviation}/feedback", + ['task_status_changed', 'task', task_link] => "#{project}/dashboard/#{abbreviation}", + ['group_membership_changed', 'general', "#{project}/groups"] => "#{project}/groups", + ['tutorial_changed', 'general', "#{project}/dashboard"] => "#{project}/tutorials", + ['portfolio_received', 'portfolio', "#{project}/dashboard"] => "#{project}/portfolio" + }.each do |(event, type, link), expected| + notification = Notification.new(user: @student, event: event, notification_type: type, message: 'x', link: link) + + assert_equal expected, notification.web_path, event + end + end + + def test_web_path_for_staff_opens_the_inbox_or_the_staff_portfolio_view + abbreviation = ERB::Util.url_encode(@task_definition.abbreviation) + inbox = "/units/#{@unit.id}/tasks/inbox/#{@student.id}/#{abbreviation}?students=all" + + %w[task_submitted task_help_requested extension_requested task_comment_created].each do |event| + notification = Notification.new(user: @tutor, event: event, notification_type: 'task', message: 'x', link: task_link) + + assert_equal inbox, notification.web_path, event + end + + portfolio = Notification.new( + user: @tutor, event: 'portfolio_submitted', notification_type: 'portfolio', message: 'x', + link: "/projects/#{@project.id}/dashboard", notifiable: @project + ) + + assert_equal "/units/#{@unit.id}/students/portfolios/#{@project.id}", portfolio.web_path + end + + def test_web_path_falls_back_to_the_link_when_the_target_is_gone + gone = Notification.new(user: @student, event: 'task_comment_created', notification_type: 'feedback', + message: 'x', link: task_link(@project, 'NOPE')) + + assert_equal task_link(@project, 'NOPE'), gone.web_path + assert_nil Notification.new(user: @student, event: 'general_event', notification_type: 'general', message: 'x').web_path + end + + def test_the_api_adds_the_ids_beside_the_existing_fields + notification = notification_for(notifiable: @task, link: task_link) + + add_auth_header_for(user: @student) + get '/api/notifications' + + assert_equal 200, last_response.status + row = JSON.parse(last_response.body).find { |json| json['id'] == notification.id } + + assert_equal task_link, row['link'], 'the existing link is unchanged' + assert_equal 'test_event', row['event'] + assert_equal @unit.id, row['unit_id'] + assert_equal @project.id, row['project_id'] + assert_equal @student.id, row['student_id'] + assert_equal @task_definition.id, row['task_definition_id'] + assert_equal @task_definition.abbreviation, row['task_definition_abbr'] + assert_equal @task.id, row['task_id'] + assert row.key?('comment_id') + assert row.key?('group_id') + end +end diff --git a/test/models/notification_task_comment_test.rb b/test/models/notification_task_comment_test.rb new file mode 100644 index 0000000000..b5939f2542 --- /dev/null +++ b/test/models/notification_task_comment_test.rb @@ -0,0 +1,156 @@ +require 'test_helper' +require 'minitest/mock' + +# EN-E01: posting a task comment notifies the other party. +class NotificationTaskCommentTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @tutor = @project.tutor_for(@task_definition) + end + + # The notification email is multipart, and Mail::Body#to_s is empty for a + # multipart body. Reading it the naive way makes every refute_includes pass + # for the wrong reason, so decode the parts instead. + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + mail.multipart? ? mail.parts.map { |part| part.body.decoded }.join("\n") : mail.body.decoded + end + + def test_a_tutor_comment_notifies_the_student + assert_difference 'Notification.count', 1 do + @task.add_text_comment(@tutor, 'Have a look at question three.') + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_equal 'feedback', notification.notification_type + assert_equal 'task_comment_created', notification.event + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}/feedback" + ) + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + # The abbreviation goes into a url segment and nothing validates its format, so + # a space in it produced a link both allowlists rejected. The bell and the push + # banner dropped the person on /notifications with no task open, while the + # email opened the task, because the mail template applies no allowlist. + def test_an_abbreviation_with_a_space_is_encoded_in_the_link + @task_definition.update!(abbreviation: 'Portfolio Reflection') + @task.reload + + @task.add_text_comment(@tutor, 'Have a look at question three.') + + notification = Notification.recent_first.first + + assert_equal "/projects/#{@project.id}/dashboard/Portfolio%20Reflection/feedback", notification.link + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/Portfolio%20Reflection/feedback" + ) + end + + def test_a_student_comment_notifies_the_tutor + assert_difference 'Notification.count', 1 do + @task.add_text_comment(@student, 'I am stuck on question three.') + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + # The recipient is the other party, never the person who commented. + assert_equal @tutor, notification.user + assert_not_equal @student, notification.user + assert_equal [@tutor.email], ActionMailer::Base.deliveries.last.to + end + + def test_no_notification_when_the_feedback_preference_is_off + @student.update!(receive_feedback_notifications: false) + + assert_no_difference 'Notification.count' do + @task.add_text_comment(@tutor, 'You will not be told about this.') + end + + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_the_comment_text_is_not_in_the_notification_or_the_email + secret = 'Please do not put this sentence in an email.' + @task.add_text_comment(@tutor, secret) + NotificationEmailJob.drain + + notification = Notification.recent_first.first + body = delivered_body + + push = parsed_push_notification(notification) + + assert_not_empty body, 'guard: the body must be readable or this test proves nothing' + assert_not_includes notification.message, secret + assert_not_includes body, secret + assert_not_includes push['body'], secret + end + + def test_the_message_names_the_commenter_and_the_task + @task.add_text_comment(@tutor, 'Named check.') + + message = Notification.recent_first.first.message + + assert_includes message, @tutor.name + assert_includes message, @task_definition.abbreviation + end + + def test_the_link_points_at_the_task_feedback_on_the_student_dashboard + @task.add_text_comment(@tutor, 'Link check.') + + assert_equal( + "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}/feedback", + Notification.recent_first.first.link + ) + end + + def test_the_event_specific_template_is_used_instead_of_the_generic_one + @task.add_text_comment(@tutor, 'Template check.') + NotificationEmailJob.drain + + body = delivered_body + + # Wording that only exists in task_comment_created.*.erb. If the mailer ever + # falls back to single_notification.*.erb this fails. + assert_includes body, 'The comment is not included in this email' + end + + def test_a_notification_failure_does_not_stop_the_comment_being_posted + comment = nil + + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification exploded' } do + comment = @task.add_text_comment(@tutor, 'This must still be saved.') + end + + assert_not_nil comment + assert comment.persisted? + assert_equal 'This must still be saved.', comment.comment + end + + def test_no_notification_and_no_error_when_there_is_no_recipient + comment = TaskComment.new(recipient: nil) + + assert_no_difference 'Notification.count' do + assert_nothing_raised { @task.notify_comment_recipient(comment) } + end + end +end diff --git a/test/models/notification_task_help_requested_test.rb b/test/models/notification_task_help_requested_test.rb new file mode 100644 index 0000000000..9d10c0dc1b --- /dev/null +++ b/test/models/notification_task_help_requested_test.rb @@ -0,0 +1,149 @@ +require 'test_helper' +require 'cgi' + +# A student moving a task to Need Help notifies the responsible tutor once. +class NotificationTaskHelpRequestedTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + + # A tutor of their own, so the recipient is not just the main convenor + # that tutor_for falls back to. + tutor_role = @unit.employ_staff(FactoryBot.create(:user, :tutor), Role.tutor) + @project.enrol_in(FactoryBot.create(:tutorial, unit: @unit, campus: @project.campus, unit_role: tutor_role)) + @tutor = tutor_role.user + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + end + + def delivered_parts + mail = ActionMailer::Base.deliveries.last + + { + html: mail&.html_part&.body&.decoded.to_s, + text: mail&.text_part&.body&.decoded.to_s + } + end + + def ask_for_help(by_user: @student, **options) + @task.trigger_transition(trigger: 'need_help', by_user: by_user, **options) + end + + def test_need_help_notifies_the_tutor_once + assert_equal @tutor, @project.tutor_for(@task_definition), 'guard: the task must have its own tutor' + + assert_difference 'Notification.count', 1 do + assert ask_for_help + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + product_name = Doubtfire::Application.config.institution[:product_name] + expected_message = "#{@student.name} asked for help with #{@task_definition.name} in #{product_name}." + + assert_equal TaskStatus.need_help, @task.reload.task_status + assert_equal @tutor, notification.user + assert_equal 'task', notification.notification_type + assert_equal 'task_help_requested', notification.event + assert_equal expected_message, notification.message + assert_equal @task, notification.notifiable + + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}", + expected_body: 'A student asked for help with a task.' + ) + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@tutor.email], ActionMailer::Base.deliveries.last.to + + parts = delivered_parts + + parts.each_value do |body| + assert_not_empty body + assert_includes body, "Hi #{@tutor.first_name}" + assert_includes body, notification.web_path + end + + assert_includes parts[:text], expected_message + assert_includes parts[:html], CGI.escapeHTML(expected_message) + end + + # Staff other than the student's tutor, so it is the student-only check that + # stops this and not the rule against notifying yourself. + def test_staff_setting_need_help_does_not_ask_the_tutor_for_help + assert_difference 'Notification.count', 1 do + assert ask_for_help(by_user: @unit.main_convenor_user) + end + + # The one notification is EN-E02 telling the student about a staff change. + notification = Notification.recent_first.first + + assert_equal 'task_status_changed', notification.event + assert_equal @student, notification.user + end + + def test_an_unchanged_status_does_not_notify_again + assert ask_for_help + + assert_no_difference 'Notification.count' do + assert ask_for_help + end + end + + def test_an_internal_group_transition_does_not_notify + assert_no_difference 'Notification.count' do + assert ask_for_help(group_transition: true) + end + end + + # Uploading group work with the Need Help trigger moves every member's task, + # but only the uploader's own task should reach a tutor. + def test_a_group_upload_asking_for_help_notifies_once + unit = FactoryBot.create(:unit, student_count: 3) + group_set = GroupSet.create!(name: 'help request group set', unit: unit) + group = Group.create!(group_set: group_set, name: 'help request group', tutorial: unit.tutorials.first) + projects = unit.active_projects.first(3) + projects.each { |project| group.add_member(project) } + + task_definition = FactoryBot.create( + :task_definition, + unit: unit, + group_set: group_set, + start_date: 1.week.ago, + target_date: 1.week.from_now + ) + uploader = projects.first + uploader_task = uploader.task_for_task_definition(task_definition) + + assert_difference -> { Notification.where(event: 'task_help_requested').count }, 1 do + uploader_task.create_submission_and_trigger_state_change(uploader.student, true, nil, 'need_help') + end + + notification = Notification.where(event: 'task_help_requested').last + + assert_equal uploader.tutor_for(task_definition), notification.user + assert_equal uploader_task, notification.notifiable + + projects.each do |project| + assert_equal TaskStatus.need_help, project.task_for_task_definition(task_definition).reload.task_status, + 'guard: the upload has to reach every member task' + end + end + + def test_the_tutor_task_preference_suppresses_it + @tutor.update!(receive_task_notifications: false) + + assert_no_difference 'Notification.count' do + assert ask_for_help + end + end +end diff --git a/test/models/notification_task_status_test.rb b/test/models/notification_task_status_test.rb new file mode 100644 index 0000000000..04191a919f --- /dev/null +++ b/test/models/notification_task_status_test.rb @@ -0,0 +1,155 @@ +require 'test_helper' +require 'minitest/mock' + +# EN-E02: a staff status change notifies the student. A student's own action +# does not. +class NotificationTaskStatusTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task_definition.update!(resubmission_extensions_enabled: false) + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @tutor = @unit.main_convenor_user + + # Put the task where a tutor can mark it, then start from a clean inbox so + # the setup's own status comment does not count towards the assertions. + @task.update!(task_status: TaskStatus.ready_for_feedback) + @task.add_status_comment(@student, TaskStatus.ready_for_feedback) + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + end + + # The notification email is multipart, and Mail::Body#to_s is empty for a + # multipart body. Reading it the naive way makes every refute pass for the + # wrong reason, so decode the parts instead. + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + mail.multipart? ? mail.parts.map { |part| part.body.decoded }.join("\n") : mail.body.decoded + end + + def test_a_staff_status_change_notifies_the_student + assert_difference 'Notification.count', 1 do + assert @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_equal 'task', notification.notification_type + assert_equal 'task_status_changed', notification.event + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}" + ) + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + def test_a_students_own_action_notifies_nobody + assert_no_difference 'Notification.count' do + assert @task.trigger_transition(trigger: 'working_on_it', by_user: @student) + end + + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_an_unchanged_status_notifies_nobody + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + # Re-applying the same status is a no-op: no change, no notification. + assert_no_difference 'Notification.count' do + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + end + + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_no_notification_when_the_task_preference_is_off + @student.update!(receive_task_notifications: false) + + assert_no_difference 'Notification.count' do + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + end + + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_the_status_value_is_not_in_the_notification_or_the_email + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + NotificationEmailJob.drain + + notification = Notification.recent_first.first + body = delivered_body + + push = parsed_push_notification(notification) + + assert_not_empty body, 'guard: the body must be readable or this test proves nothing' + assert_not_includes notification.message, 'Discuss' + assert_not_includes body, 'Discuss' + assert_not_includes push['body'], 'Discuss' + end + + def test_the_message_names_the_actor_and_the_task + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + + message = Notification.recent_first.first.message + + assert_includes message, @tutor.name + assert_includes message, @task_definition.abbreviation + end + + def test_the_link_points_at_the_task_on_the_student_dashboard + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + + assert_equal( + "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}", + Notification.recent_first.first.link + ) + end + + def test_the_event_specific_template_is_used_instead_of_the_generic_one + @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + NotificationEmailJob.drain + + body = delivered_body + + # Wording that only exists in task_status_changed.*.erb. If the mailer ever + # falls back to single_notification.*.erb this fails. + assert_includes body, 'The new status is not included in this email' + end + + def test_bulk_marking_still_notifies_one_per_task + # This event ignores the bulk: flag on purpose (see the event doc): a bulk + # mark still notifies. One call, one task, one email. + assert_difference 'Notification.count', 1 do + assert @task.trigger_transition(trigger: 'discuss', by_user: @tutor, bulk: true) + end + NotificationEmailJob.drain + + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + def test_a_notification_failure_does_not_stop_the_transition + result = nil + + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification exploded' } do + result = @task.trigger_transition(trigger: 'discuss', by_user: @tutor) + end + + assert result, 'the transition must still succeed' + assert_equal TaskStatus.discuss, @task.reload.task_status + end +end diff --git a/test/models/notification_task_submitted_test.rb b/test/models/notification_task_submitted_test.rb new file mode 100644 index 0000000000..06b931134f --- /dev/null +++ b/test/models/notification_task_submitted_test.rb @@ -0,0 +1,172 @@ +require 'test_helper' +require 'cgi' +require 'minitest/mock' + +# EN-V06: a student submission notifies the responsible tutor once. +class NotificationTaskSubmittedTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task_definition.update!( + start_date: 1.week.ago, + target_date: 1.week.from_now + ) + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @tutor = @project.tutor_for(@task_definition) + end + + def delivered_parts + mail = ActionMailer::Base.deliveries.last + + { + html: mail&.html_part&.body&.decoded.to_s, + text: mail&.text_part&.body&.decoded.to_s + } + end + + def submit_for_marking(**options) + @task.trigger_transition( + trigger: 'ready_for_feedback', + by_user: @student, + **options + ) + end + + def test_ready_for_marking_notifies_the_tutor_once_without_a_status_change_event + assert_difference 'Notification.count', 1 do + assert submit_for_marking + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal TaskStatus.ready_for_feedback, @task.reload.task_status + assert_equal @tutor, notification.user + assert_equal 'task', notification.notification_type + assert_equal 'task_submitted', notification.event + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@tutor.email], ActionMailer::Base.deliveries.last.to + + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard/#{@task_definition.abbreviation}", + expected_body: 'A task is ready for marking.' + ) + end + + def test_message_and_templates_use_the_approved_tutor_facing_copy + submit_for_marking + NotificationEmailJob.drain + + notification = Notification.recent_first.first + parts = delivered_parts + product_name = Doubtfire::Application.config.institution[:product_name] + expected_message = + "#{@student.name} submitted #{@task_definition.name} for marking in #{product_name}." + + assert_equal expected_message, notification.message + assert_equal "#{product_name}: Task submitted", ActionMailer::Base.deliveries.last.subject + + parts.each_value do |body| + assert_not_empty body + assert_includes body, "Hi #{@tutor.first_name}" + assert_includes body, 'The submission and any assessment content are not included in this email.' + assert_includes body, notification.web_path + assert_includes body, '/edit_profile' + end + + assert_includes parts[:text], expected_message + assert_includes parts[:html], CGI.escapeHTML(expected_message) + end + + def test_missing_tutor_is_safely_ignored + @task.update!(task_status: TaskStatus.ready_for_feedback) + + @project.stub :tutor_for, nil do + assert_no_difference 'Notification.count' do + assert_nothing_raised do + @task.notify_tutor_of_student_request( + @student, + :student, + TaskStatus.not_started.id, + false + ) + end + end + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_tutor_task_preference_suppresses_the_notification + @tutor.update!(receive_task_notifications: false) + + assert_no_difference 'Notification.count' do + assert submit_for_marking + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_repeating_ready_for_marking_does_not_notify_again + assert submit_for_marking + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + assert_no_difference 'Notification.count' do + assert submit_for_marking + end + + assert_empty ActionMailer::Base.deliveries + end + + # A resubmission is the same move into Ready for Feedback, so the tutor hears + # about it again once they have asked for changes. + def test_resubmitting_after_fix_and_resubmit_notifies_the_tutor_again + assert submit_for_marking + assert @task.trigger_transition(trigger: 'fix_and_resubmit', by_user: @tutor) + + assert_difference -> { Notification.where(user: @tutor, event: 'task_submitted').count }, 1 do + assert submit_for_marking + end + end + + def test_internal_group_transition_does_not_amplify_the_notification + assert_no_difference 'Notification.count' do + assert submit_for_marking(group_transition: true) + end + + assert_empty ActionMailer::Base.deliveries + end + + def test_a_tutor_ready_for_feedback_transition_only_raises_the_existing_status_event + assert_difference 'Notification.count', 1 do + assert @task.trigger_transition(trigger: 'ready_for_feedback', by_user: @tutor) + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal 'task_status_changed', notification.event + assert_equal @student, notification.user + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + end + + def test_notification_failure_does_not_stop_the_submission_transition + result = nil + + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification exploded' } do + result = submit_for_marking + end + + assert result, 'the submission transition must still succeed' + assert_equal TaskStatus.ready_for_feedback, @task.reload.task_status + end +end diff --git a/test/models/notification_test.rb b/test/models/notification_test.rb new file mode 100644 index 0000000000..f59ad19358 --- /dev/null +++ b/test/models/notification_test.rb @@ -0,0 +1,239 @@ +require 'test_helper' + +class NotificationTest < ActiveSupport::TestCase + def test_the_factory_builds_a_valid_notification_for_every_category + Notification::TYPES.each do |type| + notification = FactoryBot.create(:notification, type.to_sym) + + assert notification.persisted?, "a #{type} notification did not save" + assert_equal type, notification.notification_type + assert_equal "#{type}_event", notification.event + assert notification.message.present? + end + end + + def test_an_unread_notification_is_in_the_unread_scope + notification = FactoryBot.create(:notification, :unread) + + assert_nil notification.read_at + assert_not notification.read? + assert_includes Notification.unread, notification + end + + def test_a_read_notification_is_out_of_the_unread_scope + notification = FactoryBot.create(:notification, :read) + + assert_not_nil notification.read_at + assert notification.read? + assert_not_includes Notification.unread, notification + end + + # NB-02: a notification records what it is about, and reading that thing + # clears it. Everything below is about the notifiable target. + class NotificationTargetTest < ActiveSupport::TestCase + setup do + ActionMailer::Base.deliveries.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @task_definition = @unit.task_definitions.first + @task = @project.task_for_task_definition(@task_definition) + @student = @project.student + @tutor = @project.tutor_for(@task_definition) + end + + # Exactly what the comments endpoint hands to mark_comments_as_read: the + # task's comments in load order, minus DiscussionComments. Anything looser + # would test a caller that does not exist and would hide a filtered-out + # comment whose notification therefore never clears. + # See app/api/task_comments_api.rb, GET .../comments. + def comments_the_endpoint_marks_as_read + @task.all_comments + .order('created_at ASC') + .where("TYPE is null OR TYPE != 'DiscussionComment'") + end + + def test_a_comment_notification_targets_the_comment_and_not_the_task + comment = @task.add_text_comment(@tutor, 'Have a look at question three.') + notification = Notification.recent_first.first + + assert_equal 'task_comment_created', notification.event + assert_equal comment, notification.notifiable + assert_equal 'TaskComment', notification.notifiable_type + end + + def test_a_status_change_notification_targets_the_task + @task.notify_student_of_status_change(@tutor, :tutor, -1) + notification = Notification.recent_first.first + + assert_equal 'task_status_changed', notification.event + assert_equal @task, notification.notifiable + assert_equal 'Task', notification.notifiable_type + end + + def test_reading_the_comments_clears_the_recipients_comment_notification + @task.add_text_comment(@tutor, 'Have a look at question three.') + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_nil notification.read_at, 'guard: the notification must start unread' + + @task.mark_comments_as_read(@student, comments_the_endpoint_marks_as_read) + + assert_not_nil notification.reload.read_at + end + + def test_reading_the_comments_does_not_clear_another_users_notification + @task.add_text_comment(@tutor, 'Have a look at question three.') + notification = Notification.recent_first.first + + # The tutor opening the same task must not clear the student's bell. + @task.mark_comments_as_read(@tutor, comments_the_endpoint_marks_as_read) + + assert_nil notification.reload.read_at + end + + def test_reading_the_comments_clears_a_notification_targeted_at_the_task + @task.notify_student_of_status_change(@tutor, :tutor, -1) + notification = Notification.recent_first.first + + assert_equal @task, notification.notifiable + + @task.mark_comments_as_read(@student, comments_the_endpoint_marks_as_read) + + assert_not_nil notification.reload.read_at + end + + def test_a_notification_with_no_target_survives_the_comments_being_read + notification = FactoryBot.create(:notification, :general, user: @student) + + assert_nil notification.notifiable + + @task.add_text_comment(@tutor, 'Have a look at question three.') + @task.mark_comments_as_read(@student, comments_the_endpoint_marks_as_read) + + assert_nil notification.reload.read_at + end + + def test_an_already_read_notification_keeps_its_original_read_at + comment = @task.add_text_comment(@tutor, 'Have a look at question three.') + notification = Notification.recent_first.first + + read_three_days_ago = 3.days.ago.change(usec: 0) + notification.update!(read_at: read_three_days_ago) + + @task.mark_comments_as_read(@student, comments_the_endpoint_marks_as_read) + + assert_equal comment, notification.notifiable + assert_equal read_three_days_ago.to_i, notification.reload.read_at.to_i + end + + # The staff path. A tutor's task_submitted notification is about the task, + # and the tutor opening that task to mark it is the read that clears it. + # + # The reload is load bearing. Task delegates target_date to its task + # definition, and a definition already loaded through the association keeps + # the factory's dates. Whether those are in the past varies by unit, so + # without the reload the submission lands on time_exceeded for some units + # and no tutor is notified at all. The guard turns that into a readable + # failure instead of a missing notification. + def submit_for_marking + @task_definition.update!( + start_date: 1.week.ago, + target_date: 1.week.from_now, + due_date: 3.weeks.from_now + ) + @task.reload + @task.trigger_transition(trigger: 'ready_for_feedback', by_user: @student) + + assert_equal TaskStatus.ready_for_feedback, @task.reload.task_status, + 'guard: the submission has to actually reach ready_for_feedback' + end + + def test_a_submission_notification_targets_the_task + submit_for_marking + notification = Notification.recent_first.first + + assert_equal 'task_submitted', notification.event + assert_equal @tutor, notification.user + assert_equal @task, notification.notifiable + assert_equal 'Task', notification.notifiable_type + end + + def test_reading_the_comments_clears_the_tutors_submission_notification + submit_for_marking + notification = Notification.recent_first.first + + assert_equal @tutor, notification.user + assert_nil notification.read_at, 'guard: the notification must start unread' + + @task.mark_comments_as_read(@tutor, comments_the_endpoint_marks_as_read) + + assert_not_nil notification.reload.read_at + end + + # A deleted comment can never appear in the set handed to + # mark_comments_as_read again, so a notification still pointing at it could + # never be cleared by reading. It goes when the comment goes. + def test_deleting_the_target_comment_takes_its_notification_with_it + comment = @task.add_text_comment(@tutor, 'Have a look at question three.') + notification = Notification.recent_first.first + + assert_equal comment, notification.notifiable + + comment.destroy! + + assert_not Notification.exists?(notification.id) + end + + # The other half of the same cascade. A task can be removed with its task + # definition, and a notification pointing at a task that no longer exists can + # never be reached, so it goes too. + def test_deleting_the_target_task_takes_its_notification_with_it + submit_for_marking + notification = Notification.recent_first.first + + assert_equal @task, notification.notifiable + + @task.destroy! + + assert_not Notification.exists?(notification.id) + end + + # An extension comment is a task comment, so it targets itself and clears + # through the same comment path rather than needing one of its own. + def assessed_extension + @unit.update!(auto_apply_extension_before_deadline: false) + @task_definition.update!(due_date: @task_definition.target_date + 2.weeks) + @task.reload + + extension = @task.apply_for_extension(@student, 'Please, I was unwell.', 1) + + assert extension.present?, 'guard: the extension request has to be created' + assert extension.assess_extension(@tutor, true), 'guard: the extension has to be assessed' + + extension + end + + def test_an_extension_notification_targets_the_extension_comment + extension = assessed_extension + notification = Notification.recent_first.first + + assert_equal 'extension_assessed', notification.event + assert_equal @student, notification.user + assert_equal extension, notification.notifiable + assert_equal 'TaskComment', notification.notifiable_type + end + + def test_reading_the_comments_clears_the_extension_notification + assessed_extension + notification = Notification.recent_first.first + + assert_nil notification.read_at, 'guard: the notification must start unread' + + @task.mark_comments_as_read(@student, comments_the_endpoint_marks_as_read) + + assert_not_nil notification.reload.read_at + end + end +end diff --git a/test/models/notification_tutorial_test.rb b/test/models/notification_tutorial_test.rb new file mode 100644 index 0000000000..77b37ec864 --- /dev/null +++ b/test/models/notification_tutorial_test.rb @@ -0,0 +1,155 @@ +require 'test_helper' +require 'minitest/mock' + +# EN-V04: notify only the affected student when an existing tutorial enrolment moves. +class NotificationTutorialTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + + @project = FactoryBot.create(:project) + @unit = @project.unit + @student = @project.student + @old_tutorial = FactoryBot.create( + :tutorial, + unit: @unit, + campus: @project.campus, + abbreviation: 'OLD_TUT', + meeting_day: 'Monday', + meeting_time: '09:00' + ) + @new_tutorial = FactoryBot.create( + :tutorial, + unit: @unit, + campus: @project.campus, + abbreviation: 'NEW_TUT', + meeting_day: 'Tuesday', + meeting_time: '14:30' + ) + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + + mail.multipart? ? mail.parts.map { |part| part.body.decoded }.join("\n") : mail.body.decoded + end + + def test_moving_an_existing_enrolment_notifies_only_the_affected_student + @project.enrol_in(@old_tutorial) + other_project = FactoryBot.create(:project, unit: @unit, campus: @project.campus) + + ActionMailer::Base.deliveries.clear + + assert_difference 'Notification.count', 1 do + @project.enrol_in(@new_tutorial) + end + NotificationEmailJob.drain + + notification = Notification.recent_first.first + + assert_equal @student, notification.user + assert_not_equal other_project.student, notification.user + assert_equal 'general', notification.notification_type + assert_equal 'tutorial_changed', notification.event + assert_equal 1, ActionMailer::Base.deliveries.count + assert_equal [@student.email], ActionMailer::Base.deliveries.last.to + assert_valid_push_payload( + notification, + expected_link: "/projects/#{@project.id}/dashboard", + expected_body: 'Your tutorial details changed.' + ) + end + + def test_message_and_templates_name_only_the_new_tutorial_schedule + @project.enrol_in(@old_tutorial) + + ActionMailer::Base.deliveries.clear + @project.enrol_in(@new_tutorial) + NotificationEmailJob.drain + + notification = Notification.recent_first.first + body = delivered_body + + assert_not_empty body, 'guard: the email body must be readable' + + [notification.message, body].each do |content| + assert_includes content, @new_tutorial.abbreviation + assert_includes content, @new_tutorial.meeting_day + assert_includes content, @new_tutorial.meeting_time + assert_not_includes content, @old_tutorial.abbreviation + assert_not_includes content, @old_tutorial.meeting_day + assert_not_includes content, @old_tutorial.meeting_time + end + + mail = ActionMailer::Base.deliveries.last + assert mail.multipart? + assert_includes mail.parts.map(&:mime_type), 'text/plain' + assert_includes mail.parts.map(&:mime_type), 'text/html' + assert_includes body, 'Your tutorial has changed' + end + + def test_first_tutorial_enrolment_does_not_notify + assert_no_difference 'Notification.count' do + @project.enrol_in(@new_tutorial) + end + + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_selecting_the_same_tutorial_again_does_not_notify + @project.enrol_in(@old_tutorial) + + ActionMailer::Base.deliveries.clear + + assert_no_difference 'Notification.count' do + @project.enrol_in(@old_tutorial) + end + + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_collapsing_multiple_stream_enrolments_does_not_notify + stream_one = FactoryBot.create(:tutorial_stream, unit: @unit) + stream_two = FactoryBot.create(:tutorial_stream, unit: @unit) + streamed_tutorial_one = FactoryBot.create( + :tutorial, + unit: @unit, + campus: @project.campus, + tutorial_stream: stream_one + ) + streamed_tutorial_two = FactoryBot.create( + :tutorial, + unit: @unit, + campus: @project.campus, + tutorial_stream: stream_two + ) + + @project.enrol_in(streamed_tutorial_one) + @project.enrol_in(streamed_tutorial_two) + assert_equal 2, @project.tutorial_enrolments.count + + ActionMailer::Base.deliveries.clear + + assert_no_difference 'Notification.count' do + @project.enrol_in(@new_tutorial) + end + + assert_equal [@new_tutorial], @project.reload.tutorial_enrolments.map(&:tutorial) + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_notification_failure_does_not_stop_the_tutorial_move + @project.enrol_in(@old_tutorial) + + NotificationService.stub :notify, ->(**_kwargs) { raise StandardError, 'notification failed' } do + assert_nothing_raised do + @project.enrol_in(@new_tutorial) + end + end + + assert_equal @new_tutorial, @project.reload.tutorial_enrolments.first.tutorial + end +end diff --git a/test/models/notification_unit_announcement_test.rb b/test/models/notification_unit_announcement_test.rb new file mode 100644 index 0000000000..cd1dc953d7 --- /dev/null +++ b/test/models/notification_unit_announcement_test.rb @@ -0,0 +1,369 @@ +# frozen_string_literal: true + +require 'test_helper' + +# Unit Hub announcements tell the unit's students and staff when one is +# published, pinned or meaningfully changed. +class NotificationUnitAnnouncementTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + include TestHelpers::PushNotificationHelper + + PUBLISHED = 'unit_announcement_published' + UPDATED = 'unit_announcement_updated' + + def app + Rails.application + end + + # The fan-out jobs carry until_executed locks. Sidekiq's fake mode never runs + # a job a test clears, so a lock taken on push would outlive the test and turn + # the same job away in the next one. Idempotency here comes from the dedupe + # keys, which these tests check, so the locks are switched off. + teardown do + SidekiqUniqueJobs.config.enabled = @unique_jobs_enabled + end + + setup do + @unique_jobs_enabled = SidekiqUniqueJobs.config.enabled + SidekiqUniqueJobs.config.enabled = false + ActionMailer::Base.deliveries.clear + Sidekiq::Worker.clear_all + + @unit = FactoryBot.create(:unit, with_students: false, task_count: 0, code: 'SIT222') + @campus = @unit.tutorials.first.campus + @convenor = @unit.unit_roles.where(role: Role.convenor).first.user + @tutor = FactoryBot.create(:user, :tutor) + @unit.employ_staff(@tutor, Role.tutor) + + @student = FactoryBot.create(:user, :student) + @unit.enrol_student(@student, @campus) + @withdrawn = FactoryBot.create(:user, :student) + @unit.enrol_student(@withdrawn, @campus).update!(enrolled: false) + + # The unit factory also staffs the unit's tutorial, so ask for its staff. + staff_ids = @unit.unit_roles.where(role: [Role.tutor, Role.convenor]).pluck(:user_id) + @recipient_ids = (staff_ids - [@tutor.id] + [@student.id]).uniq.sort + + @other_unit = FactoryBot.create(:unit, with_students: false, task_count: 0) + @outsider = FactoryBot.create(:user, :student) + @other_unit.enrol_student(@outsider, @other_unit.tutorials.first.campus) + end + + def announce(**attributes) + defaults = { title: 'Week 3 lab', body: 'Bring your laptop to the lab.', published_at: 1.minute.ago, author: @tutor } + @unit.unit_announcements.create!(defaults.merge(attributes)) + end + + def run_jobs + UnitAnnouncementNotificationJob.drain + end + + def notified(event = PUBLISHED) + Notification.where(event: event) + end + + def test_new_users_get_the_in_app_channel_only + user = FactoryBot.create(:user, :student) + + assert user.receive_unit_hub_notifications + assert_not user.receive_unit_hub_email_notifications + assert_not user.receive_unit_hub_push_notifications + assert_not user.receive_unit_hub_session_reminders + end + + def test_publishing_tells_enrolled_students_and_staff_but_not_the_author + add_auth_header_for(user: @tutor) + post_json "/api/units/#{@unit.id}/announcements", announcement: { title: 'Week 3 lab', body: 'Bring a laptop.', published_at: 1.minute.ago.iso8601 } + assert_equal 201, last_response.status, last_response.body + + run_jobs + + assert_equal @recipient_ids, notified.pluck(:user_id).sort + notification = notified.find_by(user: @student) + announcement = UnitAnnouncement.last + + assert_equal 'unit_hub', notification.notification_type + assert_equal 'New announcement in SIT222: Week 3 lab', notification.message + assert_equal "/unit-hub?unit=#{@unit.id}&announcement=#{announcement.id}", notification.link + assert_equal announcement, notification.notifiable + assert_equal @unit.id, notification.target_ids[:unit_id] + assert_equal announcement.id, notification.target_ids[:announcement_id] + assert_nil notification.target_ids[:session_id] + assert_nil notification.target_ids[:project_id] + assert_equal notification.link, notification.web_path + end + + def test_the_unit_hub_category_switch_leaves_a_user_out + @student.update!(receive_unit_hub_notifications: false) + announce + run_jobs + + assert_not notified.exists?(user: @student) + assert notified.exists?(user: @convenor) + end + + def test_a_scheduled_announcement_waits_for_its_publication_time + announcement = announce(published_at: 2.hours.from_now) + + job = UnitAnnouncementNotificationJob.jobs.last + assert_in_delta announcement.published_at.to_f, job['at'], 1 + UnitAnnouncementNotificationJob.clear + + # Run early, as though the clock were wrong, and nothing is sent. + UnitAnnouncementNotificationJob.new.perform(announcement.id, PUBLISHED, "published-#{announcement.published_at.to_i}") + assert_equal 0, notified.count + + travel_to(announcement.published_at + 1.minute) do + UnitAnnouncementNotificationJob.new.perform(announcement.id, PUBLISHED, "published-#{announcement.published_at.to_i}") + end + assert_equal @recipient_ids.length, notified.count + end + + def test_a_rescheduled_publication_ignores_the_old_job + announcement = announce(published_at: 2.hours.from_now) + old_version = "published-#{announcement.published_at.to_i}" + announcement.update!(published_at: 1.minute.ago) + + UnitAnnouncementNotificationJob.new.perform(announcement.id, PUBLISHED, old_version) + assert_equal 0, notified.count + + run_jobs + assert_equal @recipient_ids.length, notified.count + end + + def test_drafts_expired_and_backfilled_announcements_are_not_news + announce(published_at: nil) + announce(published_at: 3.days.ago, expires_at: 1.day.ago) + announce(published_at: 2.months.ago) + run_jobs + + assert_equal 0, notified.count + end + + def test_publishing_a_draft_later_tells_the_unit + announcement = announce(published_at: nil) + run_jobs + announcement.update!(published_at: Time.current) + run_jobs + + assert_equal @recipient_ids.length, notified.count + end + + def test_a_retried_or_repeated_job_sends_nothing_twice + announcement = announce + job_args = UnitAnnouncementNotificationJob.jobs.last['args'] + run_jobs + + assert_no_difference -> { Notification.count } do + UnitAnnouncementNotificationJob.new.perform(*job_args) + announcement.update!(expires_at: 1.week.from_now) + run_jobs + end + end + + def test_pinning_republishes_and_unpinning_does_not + announcement = announce + run_jobs + + announcement.update!(pinned: true) + run_jobs + pinned = notified.where(user: @student).order(:id).last + assert_equal 2, notified.where(user: @student).count + assert_equal 'Pinned announcement in SIT222: Week 3 lab', pinned.message + + announcement.update!(pinned: false) + run_jobs + assert_equal 2, notified.where(user: @student).count + assert_equal 0, notified(UPDATED).count + end + + def test_a_meaningful_edit_tells_the_unit_after_the_debounce_window + announcement = travel_to(2.hours.ago) { announce.tap { run_jobs } } + run_jobs + + announcement.update!(body: 'The lab has moved to the library this week.') + run_jobs + + assert_equal @recipient_ids, notified(UPDATED).pluck(:user_id).sort + assert_equal 'Announcement updated in SIT222: Week 3 lab', notified(UPDATED).first.message + end + + def test_a_typo_fix_is_not_an_update + announcement = travel_to(2.hours.ago) { announce(body: 'Bring your labtop to the lab.').tap { run_jobs } } + run_jobs + + announcement.update!(body: 'Bring your laptop to the lab!') + announcement.update!(title: 'week 3 lab') + run_jobs + + assert_equal 0, notified(UPDATED).count + end + + def test_updates_are_debounced_to_one_per_announcement_per_recipient_every_30_minutes + announcement = travel_to(2.hours.ago) { announce.tap { run_jobs } } + run_jobs + + announcement.update!(body: 'The lab has moved to the library this week.') + run_jobs + announcement.update!(body: 'The lab is back in the usual room, sorry for the noise.') + run_jobs + assert_equal 1, notified(UPDATED).where(user: @student).count + + travel_to(31.minutes.from_now) do + announcement.update!(body: 'The lab is cancelled this week because of the public holiday.') + run_jobs + end + assert_equal 2, notified(UPDATED).where(user: @student).count + end + + def test_an_edit_straight_after_publishing_is_held_back + announcement = announce + run_jobs + announcement.update!(body: 'Something completely different to say about the lab.') + run_jobs + + assert_equal 0, notified(UPDATED).count + end + + def test_meaningful_edit_rules + rules = UnitHub::Notifications + + assert_not rules.meaningful_edit?('Bring your laptop.', 'bring your laptop!') + assert_not rules.meaningful_edit?('Bring your labtop.', 'Bring your laptop.') + assert_not rules.meaningful_edit?('Meet in the the lab.', 'Meet in the lab.') + assert rules.meaningful_edit?('Lab in room 204.', 'Lab in room 205.') + assert rules.meaningful_edit?('The lab is on.', 'The lab is not on.') + assert rules.meaningful_edit?('Meet in the lab.', 'Meet in the library.') + assert rules.meaningful_edit?('Lab on Monday.', 'Lab on Tuesday instead, sorry.') + end + + def test_email_and_push_follow_their_own_opt_ins + @student.update!(receive_unit_hub_email_notifications: true, receive_unit_hub_push_notifications: false) + @convenor.update!(receive_unit_hub_email_notifications: false, receive_unit_hub_push_notifications: true) + NotificationEmailJob.clear + PushNotificationDeliveryJob.clear + + announce + run_jobs + + student_notification = notified.find_by(user: @student) + convenor_notification = notified.find_by(user: @convenor) + email_ids = NotificationEmailJob.jobs.map { |job| job['args'].first } + push_ids = PushNotificationDeliveryJob.jobs.map { |job| job['args'].first } + + assert_equal [student_notification.id], email_ids + assert_equal [convenor_notification.id], push_ids + assert NotificationService.deliver_to?(@student, 'unit_hub', channel: :email) + assert_not NotificationService.deliver_to?(@student, 'unit_hub', channel: :push) + + # Switching the whole category off wins over a channel left on. + @student.update!(receive_unit_hub_notifications: false) + assert_not NotificationService.deliver_to?(@student, 'unit_hub', channel: :email) + end + + def test_a_queued_email_or_push_checks_the_channel_again + @student.update!(receive_unit_hub_email_notifications: true, receive_unit_hub_push_notifications: true) + announce + run_jobs + notification = notified.find_by(user: @student) + @student.update!(receive_unit_hub_email_notifications: false, receive_unit_hub_push_notifications: false) + + ActionMailer::Base.deliveries.clear + NotificationEmailJob.new.perform(notification.id) + assert_empty ActionMailer::Base.deliveries + + PushNotificationService.stub(:deliver, ->(_) { flunk 'push must not be sent' }) do + PushNotificationDeliveryJob.new.perform(notification.id) + end + end + + def test_the_lock_screen_does_not_show_the_title + announcement = announce(title: 'Exam results for Jordan Smith') + run_jobs + notification = notified.find_by(user: @student) + + assert_valid_push_payload( + notification, + expected_link: "/unit-hub?unit=#{@unit.id}&announcement=#{announcement.id}", + expected_body: 'There is a new announcement in your unit.' + ) + assert_not_includes PushNotificationService.payload_for(notification), 'Jordan' + end + + def test_an_inactive_unit_tells_nobody + @unit.update!(active: false) + announce + run_jobs + + assert_equal 0, notified.count + end + + def test_a_deleted_announcement_leaves_ids_that_say_so + announcement = announce + run_jobs + notification = notified.find_by(user: @student) + announcement.destroy! + + ids = notification.reload.target_ids + assert_nil ids[:announcement_id] + assert_nil ids[:unit_id] + end + + def test_the_email_uses_the_shared_layout_with_details_and_a_button + @student.update!(receive_unit_hub_email_notifications: true) + announcement = announce(body: 'Bring your laptop to the lab. ' * 20) + run_jobs + NotificationEmailJob.drain + + mail = ActionMailer::Base.deliveries.find { |delivery| delivery.to.include?(@student.email) } + assert_not_nil mail + html = mail.html_part.body.decoded + text = mail.text_part.body.decoded + url = "/unit-hub?unit=#{@unit.id}&announcement=#{announcement.id}" + + assert_includes html, 'New announcement in SIT222' + assert_includes html, 'Week 3 lab' + assert_includes html, 'ot-details' + assert_includes html, url + assert_includes html, 'Read the announcement' + assert_includes html, '/edit_profile' + assert_includes text, "/unit-hub?unit=#{@unit.id}&announcement=#{announcement.id}" + assert_includes text, '/edit_profile' + assert_operator html.scan('Bring your laptop').length, :<, 20 + end + + def test_the_updated_email_renders + @student.update!(receive_unit_hub_email_notifications: true) + announcement = travel_to(2.hours.ago) { announce.tap { run_jobs } } + run_jobs + announcement.update!(body: 'The lab has moved to the library this week.') + run_jobs + NotificationEmailJob.drain + + mail = ActionMailer::Base.deliveries.select { |delivery| delivery.to.include?(@student.email) }.last + assert_includes mail.html_part.body.decoded, 'Announcement updated in SIT222' + assert_includes mail.text_part.body.decoded, 'The lab has moved to the library' + end + + def test_the_preferences_can_be_saved_through_the_users_api + add_auth_header_for(user: @student) + put_json "/api/users/#{@student.id}", user: { + receive_unit_hub_notifications: false, + receive_unit_hub_email_notifications: true, + receive_unit_hub_push_notifications: true, + receive_unit_hub_session_reminders: true + } + assert_equal 200, last_response.status, last_response.body + + body = JSON.parse(last_response.body) + assert_equal false, body['receive_unit_hub_notifications'] + assert body['receive_unit_hub_session_reminders'] + + put_json "/api/users/#{@student.id}", user: { receive_unit_hub_notifications: nil, receive_unit_hub_email_notifications: nil } + @student.reload + assert @student.receive_unit_hub_notifications + assert_not @student.receive_unit_hub_email_notifications + end +end diff --git a/test/models/notification_unit_session_test.rb b/test/models/notification_unit_session_test.rb new file mode 100644 index 0000000000..8631eefb53 --- /dev/null +++ b/test/models/notification_unit_session_test.rb @@ -0,0 +1,273 @@ +# frozen_string_literal: true + +require 'test_helper' + +# Unit Hub sessions tell the unit when one moves or is cancelled, and remind +# people who opted in shortly before one starts. +class NotificationUnitSessionTest < ActiveSupport::TestCase + include Rack::Test::Methods + include TestHelpers::AuthHelper + include TestHelpers::JsonHelper + include TestHelpers::PushNotificationHelper + + CHANGED = 'unit_session_changed' + STARTING_SOON = 'unit_session_starting_soon' + + def app + Rails.application + end + + # The fan-out jobs carry until_executed locks. Sidekiq's fake mode never runs + # a job a test clears, so a lock taken on push would outlive the test and turn + # the same job away in the next one. Idempotency here comes from the dedupe + # keys, which these tests check, so the locks are switched off. + teardown do + SidekiqUniqueJobs.config.enabled = @unique_jobs_enabled + end + + setup do + @unique_jobs_enabled = SidekiqUniqueJobs.config.enabled + SidekiqUniqueJobs.config.enabled = false + ActionMailer::Base.deliveries.clear + Sidekiq::Worker.clear_all + + @unit = FactoryBot.create(:unit, with_students: false, task_count: 0, code: 'SIT333') + @campus = @unit.tutorials.first.campus + @convenor = @unit.unit_roles.where(role: Role.convenor).first.user + @student = FactoryBot.create(:user, :student) + @unit.enrol_student(@student, @campus) + @withdrawn = FactoryBot.create(:user, :student) + @unit.enrol_student(@withdrawn, @campus).update!(enrolled: false) + + # The unit factory also staffs the unit's tutorial. The convenor writes the + # session below, so every other member of staff hears about it. + @other_staff_ids = @unit.unit_roles.where(role: [Role.tutor, Role.convenor]).where.not(user_id: @convenor.id).pluck(:user_id) + + @start = Time.zone.parse('2030-03-05 17:00:00 +11:00') + @session = @unit.unit_learning_sessions.create!( + title: 'HelpHub', start_at: @start, end_at: @start + 1.hour, timezone: 'Australia/Melbourne', + location: 'Building T, room 204', published: true, author: @convenor + ) + end + + def run_jobs + UnitSessionNotificationJob.drain + end + + def notified(event = CHANGED) + Notification.where(event: event) + end + + def test_a_time_change_tells_enrolled_students_but_not_the_author + travel_to(@start - 3.days) do + @session.update!(start_at: @start + 1.hour, end_at: @start + 2.hours) + run_jobs + end + + assert_equal (@other_staff_ids + [@student.id]).sort, notified.pluck(:user_id).sort + notification = notified.find_by(user: @student) + assert_equal 'unit_hub', notification.notification_type + assert_equal 'HelpHub in SIT333 has changed. Now: Tue 5 Mar at 6:00pm AEDT, Building T, room 204.', notification.message + assert_equal "/unit-hub?unit=#{@unit.id}&session=#{@session.id}", notification.link + assert_equal @session.id, notification.target_ids[:session_id] + assert_equal @unit.id, notification.target_ids[:unit_id] + assert_nil notification.target_ids[:announcement_id] + end + + def test_other_staff_hear_about_it_when_their_category_is_on + tutor = FactoryBot.create(:user, :tutor) + @unit.employ_staff(tutor, Role.tutor) + quiet_tutor = FactoryBot.create(:user, :tutor, receive_unit_hub_notifications: false) + @unit.employ_staff(quiet_tutor, Role.tutor) + + travel_to(@start - 3.days) do + @session.update!(location: 'Library, level 2') + run_jobs + end + + assert_equal (@other_staff_ids + [@student.id, tutor.id]).sort, notified.pluck(:user_id).sort + end + + def test_a_cancellation_has_its_own_copy + travel_to(@start - 3.days) do + add_auth_header_for(user: @convenor) + delete "/api/units/#{@unit.id}/sessions/#{@session.id}" + assert_equal 200, last_response.status, last_response.body + run_jobs + end + + notification = notified.find_by(user: @student) + assert_equal 'HelpHub in SIT333 on Tue 5 Mar at 5:00pm AEDT is cancelled.', notification.message + assert_valid_push_payload( + notification, + expected_link: "/unit-hub?unit=#{@unit.id}&session=#{@session.id}", + expected_body: 'A session in your unit has changed.' + ) + end + + def test_a_weekly_cancellation_says_every_session_is_off + @session.update_columns(recurrence: 'weekly', recurrence_until: (@start + 4.weeks).to_date) # rubocop:disable Rails/SkipsModelValidations + travel_to(@start - 3.days) do + @session.reload.update!(cancelled: true) + run_jobs + end + + assert_equal 'The weekly HelpHub sessions in SIT333 are cancelled.', notified.find_by(user: @student).message + end + + def test_description_only_edits_drafts_and_past_sessions_tell_nobody + travel_to(@start - 3.days) do + @session.update!(description: 'Bring questions.') + draft = @unit.unit_learning_sessions.create!(title: 'Draft', start_at: @start, end_at: @start + 1.hour) + draft.update!(location: 'Somewhere') + run_jobs + end + travel_to(@start + 1.week) do + @session.update!(location: 'Too late') + run_jobs + end + + assert_equal 0, notified.count + end + + def test_each_version_is_sent_once_however_often_the_job_runs + travel_to(@start - 3.days) do + @session.update!(location: 'Library, level 2') + args = UnitSessionNotificationJob.jobs.last['args'] + run_jobs + UnitSessionNotificationJob.new.perform(*args) + + assert_equal 1, notified.where(user: @student).count + + @session.update!(location: 'Library, level 3') + run_jobs + end + + assert_equal 2, notified.where(user: @student).count + end + + def test_a_stale_time_change_is_dropped_once_the_session_is_cancelled + travel_to(@start - 3.days) do + @session.update!(location: 'Library, level 2') + stale = UnitSessionNotificationJob.jobs.last['args'] + UnitSessionNotificationJob.clear + @session.update!(cancelled: true) + UnitSessionNotificationJob.new.perform(*stale) + assert_equal 0, notified.count + + run_jobs + end + + assert_match(/is cancelled\.\z/, notified.find_by(user: @student).message) + end + + def test_the_change_and_cancellation_emails_render_with_details + @student.update!(receive_unit_hub_email_notifications: true) + @session.update!(join_url: 'https://teams.microsoft.com/l/meetup-join/example') + + travel_to(@start - 3.days) do + @session.update!(location: 'Library, level 2') + run_jobs + NotificationEmailJob.drain + end + changed = ActionMailer::Base.deliveries.last + html = changed.html_part.body.decoded + + assert_includes html, 'Session changed in SIT333' + assert_includes html, 'Tue 5 Mar at 5:00pm AEDT to 6:00pm' + assert_includes html, 'Library, level 2' + assert_includes html, "/unit-hub?unit=#{@unit.id}&session=#{@session.id}" + assert_includes html, '/edit_profile' + assert_includes changed.text_part.body.decoded, 'Where: Library, level 2' + + ActionMailer::Base.deliveries.clear + travel_to(@start - 2.days) do + @session.update!(cancelled: true) + run_jobs + NotificationEmailJob.drain + end + cancelled = ActionMailer::Base.deliveries.last + html = cancelled.html_part.body.decoded + + assert_includes html, 'Session cancelled in SIT333' + assert_includes html, 'This session will not run.' + assert_includes html, 'Status:</strong> Cancelled' + assert_not_includes html, 'teams.microsoft.com' + assert_includes cancelled.text_part.body.decoded, 'is cancelled.' + end + + def test_reminders_go_only_to_people_who_opted_in + @student.update!(receive_unit_hub_session_reminders: true) + other = FactoryBot.create(:user, :student) + @unit.enrol_student(other, @campus) + @withdrawn.update!(receive_unit_hub_session_reminders: true) + @convenor.update!(receive_unit_hub_session_reminders: true) + + travel_to(@start - 20.minutes) { SendUnitSessionRemindersJob.new.perform } + + assert_equal [@student.id], notified(STARTING_SOON).pluck(:user_id) + notification = notified(STARTING_SOON).first + assert_equal 'HelpHub in SIT333 starts at 5:00pm AEDT (Building T, room 204).', notification.message + assert_valid_push_payload( + notification, + expected_link: "/unit-hub?unit=#{@unit.id}&session=#{@session.id}", + expected_body: 'A session in your unit starts soon.' + ) + end + + def test_a_reminder_is_sent_once_across_runs_and_not_too_early_or_late + @student.update!(receive_unit_hub_session_reminders: true) + + travel_to(@start - 45.minutes) { SendUnitSessionRemindersJob.new.perform } + assert_equal 0, notified(STARTING_SOON).count + + [30, 25, 10, 5].each do |minutes| + travel_to(@start - minutes.minutes) { SendUnitSessionRemindersJob.new.perform } + end + assert_equal 1, notified(STARTING_SOON).count + + travel_to(@start + 5.minutes) { SendUnitSessionRemindersJob.new.perform } + assert_equal 1, notified(STARTING_SOON).count + end + + def test_each_weekly_occurrence_gets_its_own_reminder + @student.update!(receive_unit_hub_session_reminders: true) + @session.update_columns(recurrence: 'weekly', recurrence_until: (@start + 4.weeks).to_date) # rubocop:disable Rails/SkipsModelValidations + + travel_to(@start - 10.minutes) { SendUnitSessionRemindersJob.new.perform } + travel_to(@start + 1.week - 10.minutes) { SendUnitSessionRemindersJob.new.perform } + + assert_equal 2, notified(STARTING_SOON).count + end + + def test_cancelled_and_unpublished_sessions_are_not_reminded + @student.update!(receive_unit_hub_session_reminders: true) + @session.update_columns(cancelled: true) # rubocop:disable Rails/SkipsModelValidations + @unit.unit_learning_sessions.create!(title: 'Draft', start_at: @start, end_at: @start + 1.hour) + + travel_to(@start - 10.minutes) { SendUnitSessionRemindersJob.new.perform } + + assert_equal 0, notified(STARTING_SOON).count + end + + def test_the_reminder_email_renders + @student.update!(receive_unit_hub_session_reminders: true, receive_unit_hub_email_notifications: true) + + travel_to(@start - 10.minutes) do + SendUnitSessionRemindersJob.new.perform + NotificationEmailJob.drain + end + mail = ActionMailer::Base.deliveries.last + + assert_includes mail.html_part.body.decoded, 'Session starting soon in SIT333' + assert_includes mail.html_part.body.decoded, 'Building T, room 204' + assert_includes mail.html_part.body.decoded, 'session reminders are on' + assert_includes mail.text_part.body.decoded, 'When: Tue 5 Mar at 5:00pm AEDT to 6:00pm' + end + + def test_the_reminder_is_on_the_schedule + schedule = YAML.load_file(Rails.root.join('config/schedule.yml')) + + assert_equal 'SendUnitSessionRemindersJob', schedule.dig('send_unit_session_reminders', 'class') + end +end diff --git a/test/models/push_subscription_test.rb b/test/models/push_subscription_test.rb new file mode 100644 index 0000000000..6a6811af2a --- /dev/null +++ b/test/models/push_subscription_test.rb @@ -0,0 +1,140 @@ +require 'test_helper' + +# The endpoint arrives from the browser and PushNotificationService later makes +# an outbound POST to it, so anything that is not a real push service URL has to +# be refused on the way in. +class PushSubscriptionTest < ActiveSupport::TestCase + setup do + @user = FactoryBot.create(:user, :student) + end + + def build_with(endpoint) + FactoryBot.build(:push_subscription, user: @user, endpoint: endpoint) + end + + # Every service we actually expect to see, so a future change to the list + # cannot quietly drop a browser. + ACCEPTED = [ + 'https://fcm.googleapis.com/fcm/send/abc123', + 'https://android.googleapis.com/gcm/send/abc123', + 'https://updates.push.services.mozilla.com/wpush/v2/abc123', + 'https://web.push.apple.com/abc123', + 'https://webcourier.push.apple.com/abc123', + 'https://par02p.notify.windows.com/w/?token=abc123', + 'https://wns2-by3p.push.services.microsoft.com/w/?token=abc123' + ].freeze + + ACCEPTED.each_with_index do |endpoint, index| + define_method("test_accepts_known_push_service_#{index}") do + subscription = build_with(endpoint) + + assert subscription.valid?, "#{endpoint} should be accepted but was rejected with #{subscription.errors.full_messages}" + end + end + + # The SSRF cases. Each of these is a host an attacker would want the api to + # make a request to on their behalf. + REJECTED = { + 'plain http' => 'http://fcm.googleapis.com/fcm/send/abc', + 'localhost' => 'https://localhost/fcm/send/abc', + 'loopback ip' => 'https://127.0.0.1/fcm/send/abc', + 'link local metadata' => 'https://169.254.169.254/latest/meta-data/', + 'private range' => 'https://10.0.0.5/internal', + 'the api container itself' => 'https://doubtfire-api:3000/api/users', + 'an arbitrary host' => 'https://example.com/push', + 'userinfo redirect trick' => 'https://fcm.googleapis.com@evil.example.com/push', + 'non standard port' => 'https://fcm.googleapis.com:8080/fcm/send/abc', + 'suffix lookalike' => 'https://evil-notify.windows.com/w/?token=abc', + 'apple suffix without boundary' => 'https://evilpush.apple.com/abc', + 'apple suffix followed by another domain' => 'https://web.push.apple.com.evil.example/abc', + 'bare apple parent domain' => 'https://push.apple.com/abc', + 'host substring lookalike' => 'https://fcm.googleapis.com.evil.example.com/push', + 'not a url at all' => 'not a url', + 'file scheme' => 'file:///etc/passwd' + }.freeze + + REJECTED.each do |name, endpoint| + define_method("test_rejects_#{name.tr(' ', '_')}") do + subscription = build_with(endpoint) + + assert_not subscription.valid?, "#{endpoint} (#{name}) should have been rejected" + assert_includes subscription.errors[:endpoint].join, 'recognised push service' + end + end + + def test_the_factory_endpoint_is_accepted + # Guards against the allowlist and the factory drifting apart, which would + # break every other push test at once and look like an unrelated failure. + assert FactoryBot.build(:push_subscription, user: @user).valid? + end + + def test_push_service_endpoint_predicate_handles_blank_input + assert_not PushSubscription.push_service_endpoint?(nil) + assert_not PushSubscription.push_service_endpoint?('') + end + + def test_an_endpoint_is_still_required + subscription = build_with(nil) + + assert_not subscription.valid? + assert_includes subscription.errors[:endpoint].join, "can't be blank" + end + + # DN-30: the keys must be usable web push material, not merely present and + # under 255 characters. A malformed key passes the length validation and then + # fails deep in web-push encryption, poisoning the fan-out for the user. + def test_accepts_padded_keys_from_the_factory + assert FactoryBot.build(:push_subscription, user: @user).valid? + end + + def test_accepts_unpadded_keys + # The same key pair as the factory, with the base64 padding stripped, which + # is how a browser's PushSubscription usually presents it. + subscription = FactoryBot.build( + :push_subscription, + user: @user, + p256dh: 'BJy8RpjMkwOPDIIXSu-FTe7OosAwY9G86_evhrn0jJbPnoxXjBYpn7aPHEIaRh3GxCzFvwYXjKWvtu3FEMaBQMY', + auth: 'CUkmaYqq8eINt1HTnFY65w' + ) + + assert subscription.valid?, subscription.errors.full_messages.join(', ') + end + + def test_rejects_a_non_base64_p256dh + subscription = FactoryBot.build(:push_subscription, user: @user, p256dh: 'not valid base64 !!') + + assert_not subscription.valid? + assert_not_empty subscription.errors[:p256dh] + end + + def test_rejects_a_wrong_length_p256dh + # 'AAAA' is valid base64 but decodes to 3 bytes, not the 65 a prime256v1 + # public key needs. + subscription = FactoryBot.build(:push_subscription, user: @user, p256dh: 'AAAA') + + assert_not subscription.valid? + assert_not_empty subscription.errors[:p256dh] + end + + def test_rejects_a_wrong_length_auth + # Decodes to 3 bytes, not the 16 the auth secret must be. + subscription = FactoryBot.build(:push_subscription, user: @user, auth: 'AAAA') + + assert_not subscription.valid? + assert_not_empty subscription.errors[:auth] + end + + def test_valid_web_push_key_predicate_rejects_blank_and_garbage + assert_not PushSubscription.valid_web_push_key?(nil, PushSubscription::AUTH_BYTES) + assert_not PushSubscription.valid_web_push_key?('', PushSubscription::AUTH_BYTES) + assert_not PushSubscription.valid_web_push_key?('%%%not-base64%%%', PushSubscription::AUTH_BYTES) + end + + def test_rejects_a_correct_length_key_that_is_not_a_curve_point + invalid_point = Base64.urlsafe_encode64("\x04" + ("\x42" * 64)) + subscription = FactoryBot.build(:push_subscription, user: @user, p256dh: invalid_point) + + assert_not subscription.valid? + assert_not_empty subscription.errors[:p256dh] + end +end diff --git a/test/models/task_definition_test.rb b/test/models/task_definition_test.rb index 6312419108..c4939503f8 100644 --- a/test/models/task_definition_test.rb +++ b/test/models/task_definition_test.rb @@ -187,10 +187,20 @@ def test_group_tasks group_set.save! path = Rails.root.join('test_files', 'unit_csv_imports', 'import_group_tasks.csv') - u.import_tasks_from_csv File.new(path) + assert_difference( + -> { NewTaskAvailableNotificationJob.jobs.size }, + 1 + ) do + u.import_tasks_from_csv File.new(path) + end assert_equal 1, group_set.task_definitions.count assert_equal initial_count + 1, u.task_definitions.count + assert_not_nil group_set.task_definitions.first.new_task_notifications_from + + assert_no_difference -> { NewTaskAvailableNotificationJob.jobs.size } do + u.import_tasks_from_csv File.new(path) + end end def test_export_task_definitions_csv diff --git a/test/models/unit_model_test.rb b/test/models/unit_model_test.rb index b53659ea02..0da1d2eaa7 100644 --- a/test/models/unit_model_test.rb +++ b/test/models/unit_model_test.rb @@ -200,7 +200,13 @@ def test_rollover_of_learning_summary @unit.draft_task_definition = lsr @unit.save - unit2 = @unit.rollover TeachingPeriod.find(2), nil, nil, nil + unit2 = nil + assert_difference( + -> { NewTaskAvailableNotificationJob.jobs.size }, + @unit.task_definitions.count + ) do + unit2 = @unit.rollover TeachingPeriod.find(2), nil, nil, nil + end assert_not_nil unit2.draft_task_definition refute_equal lsr, unit2.draft_task_definition diff --git a/test/services/notification_delivery_policy_concurrency_test.rb b/test/services/notification_delivery_policy_concurrency_test.rb new file mode 100644 index 0000000000..7a6057231e --- /dev/null +++ b/test/services/notification_delivery_policy_concurrency_test.rb @@ -0,0 +1,186 @@ +# frozen_string_literal: true + +require 'test_helper' +require 'timeout' + +class NotificationDeliveryPolicyConcurrencyTest < ActiveSupport::TestCase + # Each producer must commit on its own connection, as real request/job + # transactions do. A fixture transaction would hide the recipient from them. + self.use_transactional_tests = false + + setup do + skip 'Requires MySQL/MariaDB repeatable-read transactions' unless ActiveRecord::Base.connection.adapter_name == 'Mysql2' + + @previous_limits = %w[ + DOUBTFIRE_NOTIFICATION_RECIPIENT_LIMIT + DOUBTFIRE_NOTIFICATION_RECIPIENT_WINDOW_SECONDS + ].index_with { |name| ENV.fetch(name, nil) } + ENV['DOUBTFIRE_NOTIFICATION_RECIPIENT_LIMIT'] = '1' + ENV['DOUBTFIRE_NOTIFICATION_RECIPIENT_WINDOW_SECONDS'] = '3600' + + nonce = SecureRandom.hex(8) + @recipient = FactoryBot.create(:user, username: "quota-race-#{nonce}", + email: "quota-race-#{nonce}@example.invalid") + NotificationEmailJob.clear + PushNotificationDeliveryJob.clear + end + + teardown do + @recipient&.destroy! + @previous_limits&.each { |name, value| value.nil? ? ENV.delete(name) : ENV[name] = value } + end + + def test_outer_transaction_snapshot_cannot_hide_a_committed_quota_reservation + first, second = reserve_after_older_snapshot + + assert_equal 0, @snapshot_notification_count + assert_equal 'pending', first.reload.email_delivery_state + assert_equal 'throttled', second.reload.email_delivery_state + assert_equal 2, Notification.where(user_id: @recipient.id).count + assert_equal([[first.id]], NotificationEmailJob.jobs.map { |job| job['args'] }) + assert_equal([[first.id]], PushNotificationDeliveryJob.jobs.map { |job| job['args'] }) + end + + def test_duplicate_lookup_sees_the_winner_despite_an_older_transaction_snapshot + first, second = reserve_after_older_snapshot(dedupe_key: 'concurrent-event') + + assert_equal 0, @snapshot_notification_count + assert_equal first.id, second.id + assert_equal 'pending', second.reload.email_delivery_state + assert_equal 1, Notification.where(user_id: @recipient.id).count + assert_equal([[first.id]], NotificationEmailJob.jobs.map { |job| job['args'] }) + assert_equal([[first.id]], PushNotificationDeliveryJob.jobs.map { |job| job['args'] }) + end + + def test_duplicate_with_a_failed_push_handoff_retries_after_outer_commit + first, second = reserve_after_older_snapshot(dedupe_key: 'retry-push', fail_first_push: true) + + assert_equal first.id, second.id + assert_not_nil second.reload.delivered_at + assert_equal [[first.id]], (NotificationEmailJob.jobs.map { |job| job['args'] }) + assert_equal [[first.id]], (PushNotificationDeliveryJob.jobs.map { |job| job['args'] }) + end + + def test_recipient_update_does_not_abort_a_reservation_in_an_older_snapshot + snapshot_established = Queue.new + recipient_update_committed = Queue.new + producers = [] + original_first_name = @recipient.first_name + snapshot_first_name = nil + + producers << Thread.new do + ActiveRecord::Base.connection_pool.with_connection do + Timeout.timeout(15) { snapshot_established.pop } + User.find(@recipient.id).update!(first_name: 'Changed while reserving') + recipient_update_committed << true + end + end + + producers << Thread.new do + ActiveRecord::Base.connection_pool.with_connection do + ActiveRecord::Base.transaction(isolation: :repeatable_read) do + # A surrounding assessment can already have a snapshot when a + # student changes their profile before the notification is reserved. + recipient = User.find(@recipient.id) + snapshot_first_name = recipient.first_name + snapshot_established << true + Timeout.timeout(15) { recipient_update_committed.pop } + NotificationService.reserve(user: recipient, type: 'extension', + event: 'resubmission_deadline_changed', + message: 'Concurrent recipient update regression') + end + end + end + + notification = Timeout.timeout(20) { producers.map(&:value).last } + assert_equal original_first_name, snapshot_first_name + assert_equal 'Changed while reserving', @recipient.reload.first_name + assert_equal 'pending', notification.reload.email_delivery_state + assert_equal 1, Notification.where(user_id: @recipient.id).count + assert_equal [[notification.id]], (NotificationEmailJob.jobs.map { |job| job['args'] }) + assert_empty PushNotificationDeliveryJob.jobs + ensure + producers&.each { |producer| producer.kill if producer.alive? } + producers&.each(&:join) + end + + def test_reservation_restores_mariadb_snapshot_setting_after_success_and_failure + connection = ActiveRecord::Base.connection + skip 'Requires MariaDB snapshot isolation setting' unless connection.mariadb? && + connection.select_rows("SHOW VARIABLES LIKE 'innodb_snapshot_isolation'").any? + + original = connection.select_value('SELECT @@SESSION.innodb_snapshot_isolation').to_i + [0, 1].each do |setting| + connection.execute("SET SESSION innodb_snapshot_isolation = #{setting}") + NotificationService.reserve(user: @recipient, type: 'general', event: 'setting_restoration', message: 'Reserved') + assert_equal setting, connection.select_value('SELECT @@SESSION.innodb_snapshot_isolation').to_i + + assert_raises(ActiveRecord::RecordInvalid) do + NotificationService.reserve(user: @recipient, type: 'general', event: 'invalid_reservation', message: nil) + end + assert_equal setting, connection.select_value('SELECT @@SESSION.innodb_snapshot_isolation').to_i + end + ensure + connection.execute("SET SESSION innodb_snapshot_isolation = #{original}") unless original.nil? + end + + def test_outer_rollback_removes_reservation_without_queuing_a_channel + assert_no_difference 'Notification.count' do + User.transaction do + notification = NotificationService.reserve(user: @recipient, type: 'general', event: 'rolled_back', message: 'Reserved') + assert notification.persisted? + raise ActiveRecord::Rollback + end + end + assert_empty NotificationEmailJob.jobs + assert_empty PushNotificationDeliveryJob.jobs + end + + private + + def reserve_after_older_snapshot(dedupe_key: nil, fail_first_push: false) + snapshot_established = Queue.new + first_reservation_committed = Queue.new + producers = [] + + producers << Thread.new do + ActiveRecord::Base.connection_pool.with_connection do + Timeout.timeout(15) { snapshot_established.pop } + notification = if fail_first_push + PushNotificationDeliveryJob.stub(:perform_async, false) do + raise_notification('first_reservation', dedupe_key) + end + else + raise_notification('first_reservation', dedupe_key) + end + # notify has returned from its transaction, so the first row and both + # channel hand-offs exist before the older transaction resumes. + first_reservation_committed << true + notification + end + end + + producers << Thread.new do + ActiveRecord::Base.connection_pool.with_connection do + ActiveRecord::Base.transaction(isolation: :repeatable_read) do + # An ordinary read in a surrounding workflow (for example the + # project's eligibility reload) establishes this same older snapshot. + @snapshot_notification_count = Notification.where(user_id: @recipient.id).count + snapshot_established << true + Timeout.timeout(15) { first_reservation_committed.pop } + raise_notification('second_reservation', dedupe_key) + end + end + end + + Timeout.timeout(20) { producers.map(&:value) } + ensure + producers&.each { |producer| producer.kill if producer.alive? } + producers&.each(&:join) + end + + def raise_notification(event, dedupe_key) + NotificationService.notify(user: User.find(@recipient.id), type: 'general', event: event, + message: 'Concurrent quota regression', dedupe_key: dedupe_key) + end +end diff --git a/test/services/notification_delivery_policy_test.rb b/test/services/notification_delivery_policy_test.rb new file mode 100644 index 0000000000..3691ece394 --- /dev/null +++ b/test/services/notification_delivery_policy_test.rb @@ -0,0 +1,57 @@ +# frozen_string_literal: true + +require 'test_helper' + +class NotificationDeliveryPolicyTest < ActiveSupport::TestCase + def with_limits + names = %w[DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT DOUBTFIRE_NOTIFICATION_RECIPIENT_LIMIT DOUBTFIRE_NOTIFICATION_RECIPIENT_WINDOW_SECONDS] + previous = names.index_with { |name| ENV.fetch(name, nil) } + ENV[names[0]] = '2' + ENV[names[1]] = '2' + ENV[names[2]] = '3600' + yield + ensure + previous.each { |name, value| value.nil? ? ENV.delete(name) : ENV[name] = value } + end + + def test_fanout_boundary_and_explicit_override + with_limits do + [1, 2].each do |count| + assert NotificationDeliveryPolicy.fanout_allowed?(event: 'test', trigger: 'unit:1', recipient_count: count) + end + assert_not NotificationDeliveryPolicy.fanout_allowed?(event: 'test', trigger: 'unit:1', recipient_count: 3) + assert NotificationDeliveryPolicy.fanout_allowed?(event: 'test', trigger: 'unit:1', recipient_count: 3, allow_large_fanout: true) + assert_not NotificationDeliveryPolicy.fanout_allowed?(event: 'test', trigger: 'unit:1', recipient_count: 3, allow_large_fanout: 'true') + end + end + + def test_burst_preserves_in_app_notifications_without_external_delivery + user = FactoryBot.create(:user) + with_limits do + notifications = 3.times.map do + NotificationService.notify(user: user, type: 'general', event: 'burst', message: 'Safe test message') + end + assert_equal %w[pending pending throttled], notifications.map(&:email_delivery_state) + assert_nil notifications.last.delivered_at + assert_equal 2, NotificationEmailJob.jobs.size + assert_equal 2, PushNotificationDeliveryJob.jobs.size + assert_equal 3, Notification.where(user: user, event: 'burst').count + + travel 3601.seconds do + notification = NotificationService.notify(user: user, type: 'general', event: 'later', message: 'Later') + assert_equal 'pending', notification.email_delivery_state + end + end + end + + def test_deduplication_does_not_consume_another_quota_slot + user = FactoryBot.create(:user) + with_limits do + twice = 2.times.map do + NotificationService.notify(user: user, type: 'general', event: 'same', message: 'Same', dedupe_key: 'same') + end + assert_equal twice.first.id, twice.last.id + assert_not NotificationDeliveryPolicy.throttled?(user) + end + end +end diff --git a/test/services/notification_service_test.rb b/test/services/notification_service_test.rb new file mode 100644 index 0000000000..a336a49f2d --- /dev/null +++ b/test/services/notification_service_test.rb @@ -0,0 +1,375 @@ +require 'test_helper' +require 'minitest/mock' + +class NotificationServiceTest < ActiveSupport::TestCase + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + PushNotificationDeliveryJob.clear + end + + def test_ambiguous_queue_failure_does_not_overwrite_completed_worker_delivery + notification = FactoryBot.create(:notification) + ambiguous_enqueue = lambda do |_id| + Notification.find(notification.id).update!(email_delivery_state: 'delivered', email_delivered_at: Time.current) + raise IOError, 'Queue reply was lost' + end + NotificationEmailJob.stub(:perform_async, ambiguous_enqueue) do + assert_equal false, NotificationService.queue_email(notification) + end + assert_equal 'delivered', notification.reload.email_delivery_state + end + + def test_notify_creates_a_notification_and_queues_id_only_channel_jobs + user = FactoryBot.create(:user) + notification = nil + + assert_difference(-> { NotificationEmailJob.jobs.size }, 1) do + assert_difference(-> { PushNotificationDeliveryJob.jobs.size }, 1) do + notification = NotificationService.notify( + user: user, + type: 'task', + event: 'task_comment_created', + message: 'Your tutor commented on your task.', + link: "/projects/#{user.id}" + ) + end + end + + assert notification.persisted? + assert_equal 'task', notification.notification_type + assert_equal 'task_comment_created', notification.event + + job = NotificationEmailJob.jobs.last + assert_equal 'NotificationEmailJob', job['class'] + assert_equal 'mailers', job['queue'] + assert_equal [notification.id], job['args'] + + push_job = PushNotificationDeliveryJob.jobs.last + assert_equal 'PushNotificationDeliveryJob', push_job['class'] + assert_equal 'notifications', push_job['queue'] + assert_equal [notification.id], push_job['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_email_is_not_queued_until_the_creating_transaction_commits + user = FactoryBot.create(:user) + notification = nil + + ActiveRecord::Base.transaction do + notification = NotificationService.notify( + user: user, type: 'general', event: 'group_membership_changed', message: 'In a transaction.' + ) + + assert notification.persisted? + # A worker picking the job up here could not see the row yet, so nothing + # may be queued before the transaction commits. + assert_empty NotificationEmailJob.jobs + end + + assert_equal 1, NotificationEmailJob.jobs.size + assert_equal [notification.id], NotificationEmailJob.jobs.last['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_a_rolled_back_transaction_queues_no_email + user = FactoryBot.create(:user) + + ActiveRecord::Base.transaction do + NotificationService.notify( + user: user, type: 'general', event: 'rolled_back_event', message: 'Never happened.' + ) + raise ActiveRecord::Rollback + end + + assert_equal 0, Notification.where(event: 'rolled_back_event').count + assert_empty NotificationEmailJob.jobs + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_notify_requires_an_event_keyword + user = FactoryBot.create(:user) + + assert_raises ArgumentError do + NotificationService.notify(user: user, type: 'general', message: 'No event given.') + end + end + + def test_blank_event_is_rejected + user = FactoryBot.create(:user) + + assert_no_difference 'Notification.count' do + assert_raises ActiveRecord::RecordInvalid do + NotificationService.notify(user: user, type: 'general', event: '', message: 'Blank event.') + end + end + + assert_empty NotificationEmailJob.jobs + assert_empty PushNotificationDeliveryJob.jobs + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_a_symbol_event_is_stored_as_a_string + user = FactoryBot.create(:user) + + notification = NotificationService.notify( + user: user, type: 'general', event: :task_comment_created, message: 'Symbol event.' + ) + + assert_equal 'task_comment_created', notification.event + end + + def test_message_at_the_validated_maximum_survives_a_round_trip + user = FactoryBot.create(:user) + long_message = 'a' * 500 + + notification = NotificationService.notify( + user: user, type: 'general', event: 'long_message_check', message: long_message + ) + + # Fails before the message column became text: 500 characters passed + # validation and were then truncated or rejected by VARCHAR(255). + assert_equal 500, notification.reload.message.length + end + + def test_notification_is_suppressed_when_the_category_preference_is_off + user = FactoryBot.create(:user, receive_feedback_notifications: false) + assert_no_difference 'Notification.count' do + result = NotificationService.notify( + user: user, type: 'feedback', event: 'task_comment_created', message: 'Suppressed.' + ) + assert_nil result + end + + assert_empty NotificationEmailJob.jobs + assert_empty PushNotificationDeliveryJob.jobs + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_feedback_notification_is_queued_when_the_category_preference_is_on + user = FactoryBot.create(:user, receive_feedback_notifications: true) + notification = nil + + assert_difference(-> { NotificationEmailJob.jobs.size }, 1) do + assert_difference(-> { PushNotificationDeliveryJob.jobs.size }, 1) do + assert_difference 'Notification.count', 1 do + notification = NotificationService.notify( + user: user, type: 'feedback', event: 'feedback_available', message: 'Feedback available.' + ) + + assert notification.persisted? + end + end + end + + assert_equal [notification.id], NotificationEmailJob.jobs.last['args'] + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_reservation_rechecks_the_preference_from_the_current_recipient_row + user = FactoryBot.create(:user, receive_feedback_notifications: true) + User.find(user.id).update!(receive_feedback_notifications: false) + + assert_no_difference 'Notification.count' do + assert_nil NotificationService.notify( + user: user, type: 'feedback', event: 'task_comment_created', message: 'Suppressed.' + ) + end + assert_empty NotificationEmailJob.jobs + assert_empty PushNotificationDeliveryJob.jobs + end + + def test_task_preference_gates_notifications_in_both_directions + user = FactoryBot.create(:user, receive_task_notifications: true) + notification = nil + + assert_difference(-> { NotificationEmailJob.jobs.size }, 1) do + assert_difference(-> { PushNotificationDeliveryJob.jobs.size }, 1) do + assert_difference 'Notification.count', 1 do + notification = NotificationService.notify( + user: user, type: 'task', event: 'task_due_date_changed', message: 'Task date changed.' + ) + + assert notification.persisted? + end + end + end + assert_equal [notification.id], NotificationEmailJob.jobs.last['args'] + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + + user.update!(receive_task_notifications: false) + + assert_no_difference -> { NotificationEmailJob.jobs.size } do + assert_no_difference -> { PushNotificationDeliveryJob.jobs.size } do + assert_no_difference 'Notification.count' do + result = NotificationService.notify( + user: user, type: 'task', event: 'task_due_date_changed', message: 'Suppressed task change.' + ) + + assert_nil result + end + end + end + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_portfolio_preference_gates_notifications_in_both_directions + user = FactoryBot.create(:user, receive_portfolio_notifications: true) + notification = nil + + assert_difference(-> { NotificationEmailJob.jobs.size }, 1) do + assert_difference(-> { PushNotificationDeliveryJob.jobs.size }, 1) do + assert_difference 'Notification.count', 1 do + notification = NotificationService.notify( + user: user, type: 'portfolio', event: 'portfolio_received', message: 'Portfolio received.' + ) + + assert notification.persisted? + end + end + end + assert_equal [notification.id], NotificationEmailJob.jobs.last['args'] + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + + user.update!(receive_portfolio_notifications: false) + + assert_no_difference -> { NotificationEmailJob.jobs.size } do + assert_no_difference -> { PushNotificationDeliveryJob.jobs.size } do + assert_no_difference 'Notification.count' do + result = NotificationService.notify( + user: user, type: 'portfolio', event: 'portfolio_received', message: 'Suppressed portfolio receipt.' + ) + + assert_nil result + end + end + end + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_types_without_a_preference_are_always_queued + user = FactoryBot.create( + :user, + receive_task_notifications: false, + receive_feedback_notifications: false, + receive_portfolio_notifications: false + ) + notification = nil + + assert_difference(-> { NotificationEmailJob.jobs.size }, 1) do + assert_difference(-> { PushNotificationDeliveryJob.jobs.size }, 1) do + notification = NotificationService.notify( + user: user, type: 'general', event: 'always_sent', message: 'General notice.' + ) + end + end + + assert notification.persisted? + assert_equal [notification.id], NotificationEmailJob.jobs.last['args'] + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end + + # 'extension' has no entry in Notification::PREFERENCE_FOR_TYPE, so + # deliver_to? returns true regardless of the three category toggles. The + # event name is the one ExtensionComment actually raises. + def test_extension_notifications_are_always_queued + user = FactoryBot.create( + :user, + receive_task_notifications: false, + receive_feedback_notifications: false, + receive_portfolio_notifications: false + ) + notification = nil + + assert_difference(-> { NotificationEmailJob.jobs.size }, 1) do + assert_difference(-> { PushNotificationDeliveryJob.jobs.size }, 1) do + notification = NotificationService.notify( + user: user, type: 'extension', event: 'extension_assessed', message: 'Extension decision available.' + ) + end + end + + assert notification.persisted? + assert_equal [notification.id], NotificationEmailJob.jobs.last['args'] + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_a_queue_failure_does_not_block_the_in_app_notification + user = FactoryBot.create(:user) + notification = nil + + NotificationEmailJob.stub(:perform_async, ->(_id) { raise 'redis unavailable' }) do + notification = NotificationService.notify( + user: user, type: 'general', event: 'queue_failure_check', message: 'Still saved.' + ) + + assert notification.persisted? + end + + assert_equal 0, NotificationEmailJob.jobs.size + assert_equal 1, PushNotificationDeliveryJob.jobs.size + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + assert_not_nil notification.reload.delivered_at + end + + def test_dedupe_key_delivers_only_once + user = FactoryBot.create(:user) + attributes = { + user: user, + type: 'task', + event: 'new_task_available', + message: 'A task is available.', + dedupe_key: 'new_task_available:task-definition:123' + } + + assert_difference 'Notification.count', 1 do + first = NotificationService.notify(**attributes) + second = NotificationService.notify(**attributes) + + assert_equal first, second + assert_not_nil first.delivered_at + end + + assert_equal 1, NotificationEmailJob.jobs.size + assert_equal 1, PushNotificationDeliveryJob.jobs.size + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_failed_channel_handoff_is_retried_at_least_once + user = FactoryBot.create(:user) + attributes = { + user: user, + type: 'task', + event: 'new_task_available', + message: 'A task is available.', + dedupe_key: 'new_task_available:task-definition:456' + } + failure = ->(_notification_id) { raise StandardError, 'redis unavailable' } + + PushNotificationDeliveryJob.stub(:perform_async, failure) do + NotificationService.notify(**attributes) + end + + notification = Notification.find_by!(dedupe_key: attributes[:dedupe_key]) + assert_nil notification.delivered_at + assert_equal 1, NotificationEmailJob.jobs.size + assert_equal 0, PushNotificationDeliveryJob.jobs.size + assert_equal 0, ActionMailer::Base.deliveries.count + + assert_no_difference 'Notification.count' do + NotificationService.notify(**attributes) + end + + assert_not_nil notification.reload.delivered_at + # The after-commit email belongs to the one created row; retrying only the + # failed async push hand-off must not enqueue a duplicate email. + assert_equal 1, NotificationEmailJob.jobs.size + assert_equal 1, PushNotificationDeliveryJob.jobs.size + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end +end diff --git a/test/services/push_notification_service_test.rb b/test/services/push_notification_service_test.rb new file mode 100644 index 0000000000..b16dba29a0 --- /dev/null +++ b/test/services/push_notification_service_test.rb @@ -0,0 +1,601 @@ +require 'test_helper' +# test_helper does not pull this in, and Object#stub comes from it. Without it +# test_both_timeouts_are_passed_to_the_gem errors with "undefined method 'stub' +# for module WebPush", which it has done since it was written. +require 'minitest/mock' + +# MN-F02: the push channel actually sends. +# +# These tests go through the real web-push gem and stub the HTTP call to the +# push service, rather than stubbing WebPush itself. That is deliberate: the +# part most likely to be wired up wrong is the gem call and the payload, and a +# stub of WebPush.payload_send would prove neither. +class PushNotificationServiceTest < ActiveSupport::TestCase + # A throwaway VAPID pair, generated for these tests. The matching browser key + # pair lives in the push_subscription factory, which is where the real + # prime256v1 public key the gem needs to encrypt against now comes from. + VAPID_PUBLIC = 'BOs-KbIoHK7gUIX3i2_uEuDoouj-GKxB-mY9CRmLNmd4Wn-SSl254E1g6jR1ukL3e37p8uCpaMjOvfAB0BwzvSI='.freeze + VAPID_PRIVATE = '_NFIWSUTdCdLJJFh87pf4ekQLmNYqsweZ4288NpVZaY='.freeze + + # Fixed rather than sequenced, because every test here has to stub the exact + # URL the gem will post to. + ENDPOINT = 'https://fcm.googleapis.com/fcm/send/test-browser'.freeze + + setup do + @user = FactoryBot.create(:user, :student) + @notification = Notification.create!( + user: @user, + notification_type: 'feedback', + event: 'task_comment_created', + message: 'Andrew Cain commented on 1.1P in COS10001.', + link: '/projects/2/dashboard/1.1P' + ) + end + + # Set the keys for the block and put the environment back exactly as it was. + # + # Restoring rather than deleting matters: the development container now has + # real values in its environment, so a test that deleted them would leave the + # process different from how it found it, and a test that assumed they were + # absent to begin with would pass in CI and fail on a developer's machine. + def with_env(values) + previous = values.keys.index_with { |key| ENV.fetch(key, nil) } + values.each { |key, value| value.nil? ? ENV.delete(key) : ENV[key] = value } + yield + ensure + previous.each { |key, value| value.nil? ? ENV.delete(key) : ENV[key] = value } + end + + def with_keys(&) + with_env({ 'DOUBTFIRE_VAPID_PUBLIC_KEY' => VAPID_PUBLIC, 'DOUBTFIRE_VAPID_PRIVATE_KEY' => VAPID_PRIVATE }, &) + end + + def without_keys(&) + with_env({ 'DOUBTFIRE_VAPID_PUBLIC_KEY' => nil, 'DOUBTFIRE_VAPID_PRIVATE_KEY' => nil }, &) + end + + def create_subscription(endpoint: ENDPOINT) + FactoryBot.create(:push_subscription, user: @user, endpoint: endpoint) + end + + # Read out of the built payload rather than off tag_for, so these tests fail + # if the tag stops reaching the part that is actually sent. + def tag_for(notification) + JSON.parse(PushNotificationService.payload_for(notification))['notification']['tag'] + end + + def click_data(notification = @notification) + JSON.parse(PushNotificationService.payload_for(notification)).dig('notification', 'data') + end + + def test_nothing_is_sent_when_the_vapid_keys_are_missing + create_subscription + + # No WebMock stub is registered, so any outbound request would raise. + without_keys do + assert_not PushNotificationService.configured? + assert_nothing_raised { PushNotificationService.deliver(@notification) } + end + end + + def test_a_notification_is_pushed_to_the_subscribed_browser + create_subscription + request = stub_request(:post, ENDPOINT).to_return(status: 201) + + with_keys { PushNotificationService.deliver(@notification) } + + assert_requested request + end + + def test_every_subscribed_browser_is_pushed_to + create_subscription(endpoint: "#{ENDPOINT}-one") + create_subscription(endpoint: "#{ENDPOINT}-two") + + first = stub_request(:post, "#{ENDPOINT}-one").to_return(status: 201) + second = stub_request(:post, "#{ENDPOINT}-two").to_return(status: 201) + + with_keys { PushNotificationService.deliver(@notification) } + + assert_requested first + assert_requested second + end + + def test_nothing_is_sent_when_the_user_has_no_browsers_registered + with_keys { assert_nothing_raised { PushNotificationService.deliver(@notification) } } + end + + def test_a_gone_subscription_is_deleted + create_subscription + stub_request(:post, ENDPOINT).to_return(status: 410) + + assert_difference 'PushSubscription.count', -1 do + with_keys { PushNotificationService.deliver(@notification) } + end + end + + def test_a_not_found_subscription_is_deleted + create_subscription + stub_request(:post, ENDPOINT).to_return(status: 404) + + assert_difference 'PushSubscription.count', -1 do + with_keys { PushNotificationService.deliver(@notification) } + end + end + + # A rate limit or an outage is temporary. Deleting on those would silently + # unsubscribe people the first time a push service had a bad day. + def test_a_temporary_push_service_failure_is_raised_for_retry_and_keeps_the_subscription + create_subscription + stub_request(:post, ENDPOINT).to_return(status: 429) + + assert_no_difference 'PushSubscription.count' do + with_keys do + assert_raises(PushNotificationService::DeliveryError) do + PushNotificationService.deliver(@notification) + end + end + end + end + + def test_a_temporary_failure_does_not_stop_the_other_browsers + failing_endpoint = "#{ENDPOINT}-unavailable" + healthy_endpoint = "#{ENDPOINT}-healthy" + create_subscription(endpoint: failing_endpoint) + create_subscription(endpoint: healthy_endpoint) + + stub_request(:post, failing_endpoint).to_return(status: 503) + healthy = stub_request(:post, healthy_endpoint).to_return(status: 201) + + with_keys do + assert_raises(PushNotificationService::DeliveryError) do + PushNotificationService.deliver(@notification) + end + end + + assert_requested healthy + assert_equal 2, @user.push_subscriptions.reload.count + end + + def test_one_dead_browser_does_not_stop_the_others + create_subscription(endpoint: "#{ENDPOINT}-dead") + create_subscription(endpoint: "#{ENDPOINT}-alive") + + stub_request(:post, "#{ENDPOINT}-dead").to_return(status: 410) + alive = stub_request(:post, "#{ENDPOINT}-alive").to_return(status: 201) + + with_keys { PushNotificationService.deliver(@notification) } + + assert_requested alive + assert_equal ["#{ENDPOINT}-alive"], @user.push_subscriptions.reload.map(&:endpoint) + end + + # Angular's ngsw-worker.js only displays a push if the payload has a top level + # "notification" key. Anything else needs a hand written service worker. + def test_the_payload_has_the_shape_angulars_service_worker_expects + payload = JSON.parse(PushNotificationService.payload_for(@notification)) + + assert payload.key?('notification'), 'ngsw-worker.js will ignore a payload without this key' + + body = payload['notification'] + assert_equal '/assets/icons/android-chrome-192x192.png', body['icon'] + assert_equal '/assets/icons/android-chrome-192x192.png', body['badge'] + + assert_equal 'Andrew Cain commented on 1.1P in COS10001.', body['body'] + assert_equal '/projects/2/dashboard/1.1P', body.dig('data', 'link') + assert_equal @notification.id, body.dig('data', 'notification_id') + assert_equal 'focusLastFocusedOrOpen', + body.dig('data', 'onActionClick', 'default', 'operation') + assert_equal '/projects/2/dashboard/1.1P', + body.dig('data', 'onActionClick', 'default', 'url') + assert_not_nil body['title'] + end + + def test_v2_events_with_sensitive_detail_use_reviewed_lock_screen_copy + sensitive_canary = 'PRIVATE-NAME-AND-SCHEDULE-7429' + expected_bodies = { + 'tutorial_changed' => 'Your tutorial details changed.', + 'group_membership_changed' => 'Your group membership changed.', + 'task_submitted' => 'A task is ready for marking.', + 'task_help_requested' => 'A student asked for help with a task.', + 'extension_requested' => 'A student asked for an extension.', + 'portfolio_submitted' => 'A student submitted a portfolio.', + 'portfolio_received' => 'Your portfolio submission was received.', + 'unit_announcement_published' => 'There is a new announcement in your unit.', + 'unit_announcement_updated' => 'An announcement in your unit was updated.', + 'unit_session_changed' => 'A session in your unit has changed.', + 'unit_session_starting_soon' => 'A session in your unit starts soon.' + } + + expected_bodies.each do |event, expected_body| + notification = Notification.create!( + user: @user, + notification_type: 'general', + event: event, + message: "#{sensitive_canary} appeared in the rich channel copy.", + link: '/notifications' + ) + + body = JSON.parse(PushNotificationService.payload_for(notification)) + .dig('notification', 'body') + + assert_equal expected_body, body, event + assert_not_includes body, sensitive_canary, event + end + end + + # DN-05: extension_assessed predates the MN-S04 lock-screen review, so it had + # no override and its raw message, which names the outcome and a precise date, + # went straight to the lock screen. A result and a date are both banned there. + def test_extension_assessed_uses_a_reviewed_lock_screen_body + notification = Notification.create!( + user: @user, + notification_type: 'extension', + event: 'extension_assessed', + message: 'Extension rejected for 1.1P, was due Mon Sep 14.', + link: '/projects/2/dashboard/1.1P' + ) + + body = JSON.parse(PushNotificationService.payload_for(notification)) + .dig('notification', 'body') + + assert_equal 'An extension request was assessed.', body + assert_not_includes body, 'rejected' + assert_not_includes body, 'Sep' + end + + def test_click_payload_preserves_every_approved_route_family + routes = [ + '/notifications', + '/projects/2/dashboard', + '/projects/2/groups', + '/projects/2/dashboard/1.1P', + '/projects/2/dashboard/1.1P/feedback', + '/projects/2/dashboard/T1.1', + '/projects/2/dashboard/HD1.2', + '/projects/2/dashboard/10.1H', + '/projects/2/dashboard/A15', + '/projects/2/dashboard/TASK1', + '/projects/2/dashboard/P-2.21', + '/projects/2/dashboard/D-9.568', + '/projects/2/dashboard/C-4.602', + # An abbreviation with a space in it, encoded at the point the link is + # built. This is the case the whole ticket is about. + '/projects/2/dashboard/Portfolio%20Reflection', + # 43 characters, the length this repo's own tests already create. + "/projects/2/dashboard/#{'a' * 43}", + # The Unit Hub, opened on one announcement or one session. + '/unit-hub?unit=4&announcement=12', + '/unit-hub?unit=4&session=9' + ] + + routes.each do |route| + @notification.link = route + data = click_data + + assert_equal route, data['link'] + assert_equal route, data.dig('onActionClick', 'default', 'url') + assert_equal 'focusLastFocusedOrOpen', + data.dig('onActionClick', 'default', 'operation') + end + end + + def test_click_payload_falls_back_for_missing_malformed_external_and_encoded_links + invalid_links = [ + nil, + '', + ' ', + 'http://example.test/projects/2/dashboard', + 'https://example.test/projects/2/dashboard', + 'mailto:student@example.test', + 'javascript:alert(1)', + 'data:text/html,unsafe', + 'file:///etc/passwd', + '//example.test/projects/2/dashboard', + '\\example.test\\projects\\2', + '/projects/2\\dashboard', + '/%2f%2fexample.test', + '/%5cexample.test', + '/projects/2/dashboard/%31.1P', + '/projects/2/dashboard/1.1P%3ftoken%3dsecret', + '/%252f%252fexample.test', + '/projects/2/dashboard/%', + # An encoded slash must still fall back. Admitting %20 must not become + # admitting any escape, or a path separator walks past the anchor. + '/projects/2/dashboard/A%2FB', + '/projects/2/dashboard/A%2fB', + '/projects/0/dashboard', + '/projects/2/dashboard/', + '/projects/2/dashboard/1.1P/extra', + '/projects/2/dashboard/1.1P/feedback/extra', + '/projects/2/dashboard/1.1P?token=secret', + '/projects/2/dashboard/1.1P#feedback', + # Still here to prove the segment bound is enforced, lengthened because the + # bound moved from 32 to 128. This is the only case in this list that + # changed. + "/projects/2/dashboard/#{'A1' * 70}", + '/units/2', + '/home', + # Only the exact Unit Hub shape is allowed a query string. + '/unit-hub', + '/unit-hub?unit=4', + '/unit-hub?unit=4&announcement=12&next=//example.test', + '/unit-hub?unit=0&announcement=12', + '/unit-hub?announcement=12&unit=4', + '/unit-hub?unit=4&announcement=abc', + '/unit-hub?unit=4&task=12', + "/unit-hub?unit=4&session=9\n", + '/unit-hub?unit=4&session=9#x' + ] + + invalid_links.each do |link| + @notification.link = link + data = click_data + + assert_equal '/notifications', data['link'], "expected fallback for #{link.inspect}" + assert_equal '/notifications', data.dig('onActionClick', 'default', 'url') + end + end + + def test_an_unsafe_link_is_not_copied_into_the_push_tag + @notification.link = 'https://example.test/unsafe' + + tag = tag_for(@notification) + assert_equal "notification-#{@notification.id}", tag + assert_not_includes tag, 'example.test' + end + + def test_click_payload_preserves_bounded_task_segments_without_guessing_their_format + routes = [ + '/projects/2/dashboard/85', + '/projects/2/dashboard/Alice1', + '/projects/2/dashboard/BOB1', + '/projects/2/dashboard/1.1ALICE', + '/projects/2/dashboard/1.1BOB', + '/projects/2/dashboard/feedback1', + '/projects/2/dashboard/token123', + '/projects/2/dashboard/mark85' + ] + + routes.each do |route| + @notification.link = route + data = click_data + + assert_equal route, data['link'] + assert_equal route, data.dig('onActionClick', 'default', 'url') + end + end + + def test_two_notifications_about_the_same_thing_share_a_tag + second = Notification.create!( + user: @user, + notification_type: @notification.notification_type, + event: @notification.event, + message: 'Andrew Cain commented on 1.1P in COS10001.', + link: @notification.link + ) + + assert_equal tag_for(@notification), tag_for(second) + end + + # The half that is easy to get wrong in the other direction. A tag shared by + # unrelated notifications does not tidy anything up, it hides one behind + # another and the user never sees it. + def test_notifications_about_different_things_do_not_share_a_tag + elsewhere = Notification.create!( + user: @user, + notification_type: @notification.notification_type, + event: @notification.event, + message: 'Andrew Cain commented on 2.1P in COS10001.', + link: '/projects/2/dashboard/2.1P' + ) + + assert_not_equal tag_for(@notification), tag_for(elsewhere) + end + + # The tag names the conversation, so it cannot contain anything that changes + # between messages in it. Using the notification id would be unique every time + # and would collapse nothing at all, which is the whole ticket undone. + def test_the_tag_does_not_change_between_notifications_in_a_burst + tags = 3.times.map do |index| + burst = Notification.create!( + user: @user, + notification_type: @notification.notification_type, + event: @notification.event, + message: "Andrew Cain commented on 1.1P in COS10001. (#{index})", + link: @notification.link + ) + + tag_for(burst) + end + + # Three different ids, one tag. If the id were part of it there would be + # three, so this is the assertion that pins the id out of the tag. + assert_equal 1, tags.uniq.length, tags.inspect + end + + # The one that decides between keying on the event and keying on + # notification_type. notification_type is only the preference category, so + # task_due_date_changed, task_status_changed, new_task_available and + # task_due_soon are all `task`. Keyed on that, a status change would silently + # take the place of a deadline alert about the same task, which is the failure + # this ticket exists to avoid rather than one to introduce. + def test_different_events_in_the_same_category_do_not_share_a_tag + deadline = Notification.create!( + user: @user, + notification_type: 'task', + event: 'task_due_date_changed', + message: 'The due date for 1.1P in COS10001 has changed.', + link: @notification.link + ) + due_soon = Notification.create!( + user: @user, + notification_type: 'task', + event: 'task_due_soon', + message: '1.1P in COS10001 is due soon.', + link: @notification.link + ) + + assert_equal deadline.notification_type, due_soon.notification_type + assert_not_equal tag_for(deadline), tag_for(due_soon) + end + + # link is nullable on the api, and there is nothing else to be about. Sharing + # one empty tag between every notification that happens to have no link would + # hide all but the newest of them. + def test_a_notification_with_no_link_collapses_with_nothing + first = Notification.create!( + user: @user, notification_type: 'general', event: 'system_announcement', message: 'One' + ) + second = Notification.create!( + user: @user, notification_type: 'general', event: 'system_announcement', message: 'Two' + ) + + assert_not_equal tag_for(first), tag_for(second) + assert tag_for(first).present? + end + + # Silent replacement is the point. renotify: true puts the sound and the + # vibration back on every message in the burst the tag exists to quieten. + def test_a_replacement_does_not_buzz_again + body = JSON.parse(PushNotificationService.payload_for(@notification))['notification'] + + assert_equal false, body['renotify'] + end + + # MN-D01 and MN-D05 own the wording. This ticket only adds the tag, so a + # change to either of those here is somebody else's work being overwritten. + def test_the_title_and_body_are_left_alone + body = JSON.parse(PushNotificationService.payload_for(@notification))['notification'] + + assert_equal 'Andrew Cain commented on 1.1P in COS10001.', body['body'] + assert_equal Doubtfire::Application.config.institution[:product_name], body['title'] + end + + def test_a_long_message_is_trimmed_rather_than_rejected_by_the_push_service + @notification.update!(message: 'a' * 500) + + body = JSON.parse(PushNotificationService.payload_for(@notification))['notification']['body'] + + assert_operator body.length, :<=, PushNotificationService::MAX_BODY_LENGTH + end + + # The shared fan-out queues an ID-only job, and the worker calls this service + # with no per-event push wiring. + def test_raising_a_notification_through_the_hub_queues_and_delivers_a_push + create_subscription + request = stub_request(:post, ENDPOINT).to_return(status: 201) + notification = NotificationService.notify( + user: @user, + type: 'feedback', + event: 'task_comment_created', + message: 'Raised through the hub.', + link: '/projects/2/dashboard/1.1P' + ) + + assert_equal [notification.id], PushNotificationDeliveryJob.jobs.last['args'] + assert_not_requested request + + with_keys { PushNotificationDeliveryJob.new.perform(notification.id) } + assert_requested request + end + + # Rows written before PushSubscription validated the endpoint were never + # checked, so the service refuses at send time as well. This is the line that + # actually stops a stored bad endpoint being used, so it is tested by writing + # a row that skips validation, the way an old row would look. + def test_a_stored_endpoint_that_is_not_a_push_service_is_never_requested + subscription = FactoryBot.build(:push_subscription, user: @user, endpoint: 'https://169.254.169.254/latest/meta-data/') + subscription.save!(validate: false) + + # No WebMock stub is registered for that host, so an outbound request would + # raise rather than pass silently. + with_keys { PushNotificationService.deliver(@notification) } + + assert_not_requested :post, 'https://169.254.169.254/latest/meta-data/' + assert subscription.reload.persisted?, 'a refused endpoint should be left alone, not treated as dead' + end + + def test_a_refused_endpoint_does_not_stop_the_other_browsers + FactoryBot + .build(:push_subscription, user: @user, endpoint: 'https://10.0.0.5/internal') + .save!(validate: false) + create_subscription + good = stub_request(:post, ENDPOINT).to_return(status: 201) + + with_keys { PushNotificationService.deliver(@notification) } + + assert_requested good + end + + # DN-30: a row written before the key validation existed can hold a malformed + # key. web-push would fail to encrypt against it with an error that is neither + # ExpiredSubscription nor InvalidSubscription, so the fan-out would treat it as + # transient and Sidekiq would retry, re-hitting the healthy devices. It must be + # skipped, the same as a refused endpoint. + def test_a_stored_subscription_with_malformed_keys_is_never_requested + subscription = FactoryBot.build(:push_subscription, user: @user, endpoint: ENDPOINT) + subscription.p256dh = 'not-valid-base64-key' + subscription.auth = 'nope' + subscription.save!(validate: false) + + # No WebMock stub is registered for ENDPOINT, so an outbound request would + # raise rather than pass silently. + with_keys { PushNotificationService.deliver(@notification) } + + assert_not_requested :post, ENDPOINT + assert subscription.reload.persisted?, 'a skipped row is left in place, not deleted' + end + + def test_a_subscription_with_malformed_keys_does_not_stop_the_other_browsers + bad = FactoryBot.build(:push_subscription, user: @user, endpoint: 'https://fcm.googleapis.com/fcm/send/bad-keys-browser') + bad.p256dh = 'not-valid-base64-key' + bad.auth = 'nope' + bad.save!(validate: false) + create_subscription + good = stub_request(:post, ENDPOINT).to_return(status: 201) + + assert_nothing_raised do + with_keys { PushNotificationService.deliver(@notification) } + end + + assert_requested good + end + + def test_a_stored_invalid_curve_point_does_not_stop_the_other_browsers + bad = FactoryBot.build(:push_subscription, user: @user, endpoint: 'https://fcm.googleapis.com/fcm/send/bad-point') + bad.p256dh = Base64.urlsafe_encode64("\x04" + ("\x42" * 64)) + bad.save!(validate: false) + create_subscription + good = stub_request(:post, ENDPOINT).to_return(status: 201) + + assert_nothing_raised do + with_keys { PushNotificationService.deliver(@notification) } + end + + assert_requested good + assert_not_requested :post, bad.endpoint + end + + # Timeouts are Net::HTTP settings rather than anything visible on the wire, so + # WebMock cannot see them. This one test stubs the gem instead of the HTTP + # call, which is the opposite of what the rest of this file does on purpose. + # + # It is worth the exception because web-push sets no timeouts of its own, and + # because the gem only applies read_timeout when open_timeout is also present + # (lib/web_push/request.rb line 15), so passing one without the other silently + # does nothing. + def test_delivery_limits_and_timeouts_are_passed_to_the_gem + create_subscription + captured = nil + + WebPush.stub(:payload_send, ->(**args) { captured = args }) do + with_keys { PushNotificationService.deliver(@notification) } + end + + assert_equal PushNotificationService::OPEN_TIMEOUT, captured[:open_timeout] + assert_equal PushNotificationService::READ_TIMEOUT, captured[:read_timeout] + assert_equal PushNotificationService::SSL_TIMEOUT, captured[:ssl_timeout] + assert_equal PushNotificationService::MESSAGE_TTL, captured[:ttl] + assert_equal PushNotificationService::MESSAGE_URGENCY, captured[:urgency] + end +end diff --git a/test/sidekiq/additional_notification_email_verification_job_test.rb b/test/sidekiq/additional_notification_email_verification_job_test.rb new file mode 100644 index 0000000000..6f543654e9 --- /dev/null +++ b/test/sidekiq/additional_notification_email_verification_job_test.rb @@ -0,0 +1,109 @@ +require 'test_helper' + +class AdditionalNotificationEmailVerificationJobTest < ActiveSupport::TestCase + setup do + ActionMailer::Base.deliveries.clear + @user = FactoryBot.create(:user, email: 'primary@example.edu') + @record = AdditionalNotificationEmailService.request( + user: @user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailVerificationJob.clear + end + + def test_job_payload_contains_only_record_id_and_version + jid = AdditionalNotificationEmailVerificationJob.perform_async( + @record.id, + @record.verification_version + ) + job = AdditionalNotificationEmailVerificationJob.jobs.find { |candidate| candidate['jid'] == jid } + + assert_equal [@record.id, @record.verification_version], job['args'] + assert_not_includes job.to_json, @record.email + assert_equal 'mailers', job['queue'] + end + + def test_pending_current_version_delivers_one_safe_verification_message + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version + ) + end + + mail = ActionMailer::Base.deliveries.last + assert_equal [@record.email], mail.to + assert_empty mail.cc.to_a + assert_includes mail.body.encoded, '/verify_additional_email#token=' + end + + def test_stale_version_or_verified_record_does_not_send + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version - 1 + ) + end + + @record.update!(verified_at: Time.current) + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version + ) + end + end + + def test_a_removed_address_is_dropped_without_retrying + AdditionalNotificationEmailService.remove(user: @user) + + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version + ) + end + end + + def test_expired_record_does_not_send + @record.update!(verification_expires_at: 1.minute.ago) + + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version + ) + end + end + + def test_a_rejected_verification_email_is_retried_without_the_address_in_the_error + rejection = Class.new do + def deliver_now + raise '550 5.1.1 <secondary@example.org>: Recipient address rejected' + end + end.new + + error = AdditionalNotificationEmailMailer.stub(:verification, ->(_record) { rejection }) do + assert_raises(AdditionalNotificationEmailService::DeliveryFailed) do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version + ) + end + end + + assert_not_includes error.message, @record.email + assert_nil error.cause + end + + def test_audit_store_failure_does_not_retry_an_accepted_verification_email + AdditionalNotificationEmailService.stub(:audit!, ->(*) { raise 'audit unavailable' }) do + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + AdditionalNotificationEmailVerificationJob.new.perform( + @record.id, + @record.verification_version + ) + end + end + end +end diff --git a/test/sidekiq/execute_communication_set_job_test.rb b/test/sidekiq/execute_communication_set_job_test.rb index 8ec1afb8b3..b05d766236 100644 --- a/test/sidekiq/execute_communication_set_job_test.rb +++ b/test/sidekiq/execute_communication_set_job_test.rb @@ -1,6 +1,7 @@ # frozen_string_literal: true require 'test_helper' +require 'csv' class ExecuteCommunicationSetJobTest < ActiveSupport::TestCase def test_task_comment_action_adds_a_comment_to_each_selected_students_task @@ -54,4 +55,194 @@ def test_task_comment_action_adds_a_comment_to_each_selected_students_task assert_equal 'Please review Ada for ' + unit.code, comment_one.comment assert_equal 'Please review Grace for ' + unit.code, comment_two.comment end + + # Builds a unit with three enrolled students and a set that emails all of them. + def email_set_with_three_students + unit = FactoryBot.create( + :unit, + with_students: false, + task_count: 1, + stream_count: 0, + tutorials: 0, + outcome_count: 0, + staff_count: 1 + ) + + campus = Campus.first + projects = %w[Ada Grace Katherine].map do |first_name| + student = FactoryBot.create(:user, :student) + student.update!(first_name: first_name) + unit.enrol_student(student, campus) + end + + communication_set = unit.communication_sets.create!(name: 'Email Set', active: true) + communication_rule = communication_set.communication_rules.create!( + name: 'Email Rule', + operator: 'and', + position: 0 + ) + communication_rule.communication_actions.create!( + type: 'EmailStudentAction', + subject: 'A message about {{unit.code}}', + body: 'Hello {{student.first_name}}' + ) + + [communication_set, projects] + end + + # Replaces the mailer for the duration of the block so that the nth delivery + # raises the way an unroutable address does. + def with_delivery_failing_on(nth) + original = CommunicationsMailer.method(:communication_email) + calls = 0 + + CommunicationsMailer.define_singleton_method(:communication_email) do |**kwargs| + calls += 1 + if calls == nth + failing = Object.new + failing.define_singleton_method(:deliver_now) { raise 'mailbox unavailable' } + failing + else + original.call(**kwargs) + end + end + + yield + ensure + CommunicationsMailer.singleton_class.send(:remove_method, :communication_email) + CommunicationsMailer.define_singleton_method(:communication_email, original) + end + + # Runs the job while capturing the payload it would store, which is the record + # a convenor sees of the run. + def perform_capturing_result(communication_set_id) + job = ExecuteCommunicationSetJob.new + captured = nil + job.define_singleton_method(:store) { |payload| captured = payload } + job.perform(communication_set_id) + captured + end + + # One bad address used to abort the run, and the retry started again from the + # first student, so everyone already emailed got a second copy. + def test_one_failed_delivery_does_not_stop_the_rest_of_the_run + communication_set, projects = email_set_with_three_students + ActionMailer::Base.deliveries.clear + + result = with_delivery_failing_on(2) do + perform_capturing_result(communication_set.id) + end + + assert_equal 2, ActionMailer::Base.deliveries.count + + email_rows = result[:result][:actions].select { |row| row[:action_type] == 'EmailStudentAction' } + failed = email_rows.select { |row| row[:status] == 'failed' } + + assert_equal 1, failed.count + assert_equal 2, email_rows.count { |row| row[:status] == 'sent' } + assert_equal 'mailbox unavailable', failed.first[:reason] + assert_includes projects.map(&:id), failed.first[:project_id] + + rule = communication_set.communication_rules.first + csv = ExecuteCommunicationSetJob.new.send(:build_action_log_csv, rule, projects, email_rows) + failed_csv_row = CSV.parse(csv, headers: true).find { |row| row['status'] == 'failed' } + + assert_not_nil failed_csv_row + assert_equal( + "Failed to send email to #{failed.first[:recipient_email]}: mailbox unavailable", + failed_csv_row['details'] + ) + assert failed_csv_row['executed_at'].present?, 'a failed delivery was still attempted, so it carries a time' + end + + # The check on an over-eager rescue. Nothing about a clean run changes. + def test_a_run_with_no_failures_is_unchanged + communication_set, projects = email_set_with_three_students + ActionMailer::Base.deliveries.clear + + result = perform_capturing_result(communication_set.id) + + assert_equal 3, ActionMailer::Base.deliveries.count + + email_rows = result[:result][:actions].select { |row| row[:action_type] == 'EmailStudentAction' } + assert_equal 3, email_rows.count { |row| row[:status] == 'sent' } + assert_empty email_rows.select { |row| row[:status] == 'failed' } + assert_equal projects.length, email_rows.length + end + + # Documents honestly what this change does not fix. There is still no record of + # who has already been sent to, so running the set again mails everyone again. + # A per-recipient delivery ledger is a separate ticket. + def test_a_second_run_still_mails_everyone_again + communication_set, = email_set_with_three_students + ActionMailer::Base.deliveries.clear + + with_delivery_failing_on(2) do + perform_capturing_result(communication_set.id) + end + assert_equal 2, ActionMailer::Base.deliveries.count + + perform_capturing_result(communication_set.id) + assert_equal 5, ActionMailer::Base.deliveries.count + end + + # BGW-24: the action-log CSV must stamp each row with the time its own + # action ran, not the time the CSV was generated after the batch finished. + def test_action_log_csv_stamps_each_row_with_its_own_execution_time + unit = FactoryBot.create( + :unit, + with_students: false, + task_count: 1, + stream_count: 0, + tutorials: 0, + outcome_count: 0, + staff_count: 1 + ) + + task_definition = unit.task_definitions.first + campus = Campus.first + + student = FactoryBot.create(:user, :student) + project = unit.enrol_student(student, campus) + + communication_set = unit.communication_sets.create!(name: 'Test Set', active: true) + communication_rule = communication_set.communication_rules.create!( + name: 'Comment Rule', + operator: 'and', + position: 0 + ) + action = communication_rule.communication_actions.create!( + type: 'TaskCommentAction', + task_definition: task_definition, + body: 'Please review {{student.first_name}}' + ) + + executed = Time.utc(2026, 1, 2, 3, 4, 5) + results = [ + { + action_id: action.id, + action_type: action.type, + status: 'commented', + project_id: project.id, + task_definition_id: task_definition.id, + executed_at: executed + }, + { + action_id: action.id, + action_type: action.type, + status: 'skipped', + project_id: project.id, + reason: 'no recipient' + } + ] + + csv = ExecuteCommunicationSetJob.new.send(:build_action_log_csv, communication_rule, [project], results) + rows = CSV.parse(csv, headers: true) + + commented = rows.find { |row| row['status'] == 'commented' } + skipped = rows.find { |row| row['status'] == 'skipped' } + + assert_equal executed.iso8601, commented['executed_at'], 'an executed row should carry its own action time' + assert skipped['executed_at'].blank?, 'a skipped action has no execution time' + end end diff --git a/test/sidekiq/import_students_lti_job_test.rb b/test/sidekiq/import_students_lti_job_test.rb new file mode 100644 index 0000000000..e3f4d0a1f5 --- /dev/null +++ b/test/sidekiq/import_students_lti_job_test.rb @@ -0,0 +1,90 @@ +# frozen_string_literal: true + +require 'test_helper' + +class ImportStudentsLtiJobTest < ActiveSupport::TestCase + def test_stores_readable_error_message_when_member_import_raises + unit = create( + :unit, + with_students: false, + student_count: 0 + ) + + members = [ + { + "user_id" => "test-user", + "name" => "Test User", + "given_name" => "Test", + "family_name" => "User", + "email" => "test@example.com", + "ext_user_username" => "test-user", + "roles" => ["Learner"] + } + ] + + stored_result = nil + job = ImportStudentsLtiJob.new + + job.stub( + :user_for_asserted_identity, + ->(**_kwargs) { raise StandardError, "test import failure" } + ) do + job.stub( + :store, + ->(**kwargs) { stored_result = kwargs[:result] } + ) do + job.perform(unit.id, members) + end + end + + results = JSON.parse(stored_result) + + assert_equal 1, results["errors"].count + + message = results["errors"].first["message"] + + assert_equal "StandardError: test import failure", message + end + + def test_logs_the_exception_when_member_import_raises + unit = create( + :unit, + with_students: false, + student_count: 0 + ) + + members = [ + { + "user_id" => "test-user", + "name" => "Test User", + "given_name" => "Test", + "family_name" => "User", + "email" => "test@example.com", + "ext_user_username" => "test-user", + "roles" => ["Learner"] + } + ] + + error = StandardError.new("test import failure") + logged_error = nil + + logger = Object.new + logger.define_singleton_method(:info) { |_message| } + logger.define_singleton_method(:error) { |exception| logged_error = exception } + + job = ImportStudentsLtiJob.new + + job.stub(:logger, logger) do + job.stub( + :user_for_asserted_identity, + ->(**_kwargs) { raise error } + ) do + job.stub(:store, ->(**_kwargs) {}) do + job.perform(unit.id, members) + end + end + end + + assert_same error, logged_error + end +end diff --git a/test/sidekiq/notification_email_job_test.rb b/test/sidekiq/notification_email_job_test.rb new file mode 100644 index 0000000000..092dfe5e1a --- /dev/null +++ b/test/sidekiq/notification_email_job_test.rb @@ -0,0 +1,427 @@ +# frozen_string_literal: true + +require 'test_helper' +require 'minitest/mock' + +class NotificationEmailJobTest < ActiveSupport::TestCase + setup do + ActionMailer::Base.deliveries.clear + NotificationEmailJob.clear + AdditionalNotificationEmailDeliveryJob.clear + end + + def test_perform_delivers_the_notification_email + notification = FactoryBot.create( + :notification, + event: 'general', + message: 'EN-F03 job delivery test.' + ) + + assert_difference( + -> { ActionMailer::Base.deliveries.count }, + 1 + ) do + NotificationEmailJob.new.perform(notification.id) + end + + mail = ActionMailer::Base.deliveries.last + body = if mail.multipart? + mail.parts.map { |part| part.body.decoded }.join("\n") + else + mail.body.decoded + end + expected_subject = "#{Doubtfire::Application.config.institution[:product_name]}: New notification" + + assert_equal [notification.user.email], mail.to + assert_equal expected_subject, mail.subject + assert_includes body, notification.message + end + + def test_deleted_notification_is_skipped_after_commit_enqueue + assert_no_difference( + -> { ActionMailer::Base.deliveries.count } + ) do + assert_nil NotificationEmailJob.new.perform(-1) + end + end + + def test_disabled_mail_transport_is_suppressed_without_a_false_delivery_marker + notification = FactoryBot.create(:notification) + NotificationsMailer.stub(:perform_deliveries, false) do + assert_no_difference(-> { ActionMailer::Base.deliveries.size }) do + NotificationEmailJob.new.perform(notification.id) + end + end + assert_equal 'suppressed', notification.reload.email_delivery_state + assert_nil notification.email_delivered_at + assert_equal 0, notification.email_delivery_attempts + end + + def test_notification_mail_does_not_swallow_transport_errors + notification = FactoryBot.create(:notification) + ActionMailer::Base.stub(:raise_delivery_errors, false) do + assert NotificationsMailer.single_notification(notification).message.raise_delivery_errors + end + end + + def test_delivery_state_survives_transient_failure_and_records_success + notification = FactoryBot.create(:notification) + failing = Object.new + def failing.deliver_now + raise Net::ReadTimeout + end + NotificationsMailer.stub(:single_notification, failing) do + assert_raises(Net::ReadTimeout) { NotificationEmailJob.new.perform(notification.id) } + end + assert_equal 'retrying', notification.reload.email_delivery_state + assert_equal 1, notification.email_delivery_attempts + NotificationEmailJob.new.perform(notification.id) + assert_equal 'delivered', notification.reload.email_delivery_state + assert_equal 2, notification.email_delivery_attempts + assert_not_nil notification.email_delivered_at + assert_nil notification.email_delivery_error_class + assert_no_difference(-> { ActionMailer::Base.deliveries.size }) do + NotificationEmailJob.new.perform(notification.id) + end + end + + def test_permanent_failure_is_recorded_and_dead_lettered_without_retry + notification = FactoryBot.create(:notification) + failing = Object.new + def failing.deliver_now + raise Net::SMTPFatalError, '550 rejected' + end + error = NotificationsMailer.stub(:single_notification, failing) do + assert_raises(Net::SMTPFatalError) { NotificationEmailJob.new.perform(notification.id) } + end + assert_equal 'failed', notification.reload.email_delivery_state + assert_equal 'Net::SMTPFatalError', notification.email_delivery_error_class + assert_equal :kill, NotificationEmailJob.sidekiq_retry_in_block.call(0, error) + healthy = FactoryBot.create(:notification) + NotificationEmailJob.new.perform(healthy.id) + assert_equal 'delivered', healthy.reload.email_delivery_state + end + + def test_exhausted_retries_are_persisted_after_the_dead_set_is_cleared + notification = FactoryBot.create(:notification) + NotificationEmailJob.sidekiq_retries_exhausted_block.call({ 'args' => [notification.id] }, Net::ReadTimeout.new) + assert_equal 'failed', notification.reload.email_delivery_state + assert_equal 'Net::ReadTimeout', notification.email_delivery_error_class + end + + def test_the_job_runs_on_the_mailers_queue + # Student facing email must not queue behind PDF and CSV work on default. + # The worker has to be listening on this queue, see doubtfire-deploy#10. + assert_equal 'mailers', NotificationEmailJob.get_sidekiq_options['queue'].to_s + end + + def test_no_delivery_when_the_preference_was_turned_off_after_queueing + user = FactoryBot.create(:user, receive_feedback_notifications: true) + notification = FactoryBot.create( + :notification, + :feedback, + user: user, + event: 'task_comment_created', + message: 'Queued while the category was still on.' + ) + + # retry: 3 means the job can run well after it was queued. + user.update!(receive_feedback_notifications: false) + + assert_no_difference( + -> { ActionMailer::Base.deliveries.count } + ) do + NotificationEmailJob.new.perform(notification.id) + end + end + + def test_a_type_without_a_preference_is_still_delivered + user = FactoryBot.create( + :user, + receive_task_notifications: false, + receive_feedback_notifications: false, + receive_portfolio_notifications: false + ) + notification = FactoryBot.create( + :notification, + user: user, + event: 'general', + message: 'General notices ignore the category toggles.' + ) + + assert_difference( + -> { ActionMailer::Base.deliveries.count }, + 1 + ) do + NotificationEmailJob.new.perform(notification.id) + end + end + + def test_delivery_failure_is_raised_so_sidekiq_can_retry + notification = FactoryBot.create( + :notification, + event: 'general' + ) + failing_delivery = Class.new do + def deliver_now + raise 'smtp unavailable' + end + end.new + + NotificationsMailer.stub(:single_notification, ->(_notification) { failing_delivery }) do + error = assert_raises(RuntimeError) do + NotificationEmailJob.new.perform(notification.id) + end + assert_equal 'smtp unavailable', error.message + end + end + + def test_async_payload_contains_only_the_notification_id + notification = FactoryBot.create( + :notification, + event: 'general' + ) + jid = NotificationEmailJob.perform_async(notification.id) + job = NotificationEmailJob.jobs.find do |candidate| + candidate['jid'] == jid + end + + assert_not_nil job + assert_equal 'NotificationEmailJob', job['class'] + assert_equal 'mailers', job['queue'] + assert_equal [notification.id], job['args'] + assert_equal 0, ActionMailer::Base.deliveries.count + end + + def test_verified_additional_address_receives_a_separate_copy_not_a_cc + user = FactoryBot.create(:user, email: 'primary@example.edu') + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create(:notification, user: user, event: 'general') + + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + NotificationEmailJob.new.perform(notification.id) + end + + copy_job = AdditionalNotificationEmailDeliveryJob.jobs.last + assert_equal [notification.id, additional.id, additional.verification_version], copy_job['args'] + assert_not_includes copy_job.to_json, additional.email + + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + AdditionalNotificationEmailDeliveryJob.new.perform(*copy_job['args']) + end + + primary, copy = ActionMailer::Base.deliveries.last(2) + assert_equal [user.email], primary.to + assert_equal [additional.email], copy.to + assert_empty primary.cc.to_a + assert_empty copy.cc.to_a + end + + def test_pending_or_removed_address_receives_no_copy + user = FactoryBot.create(:user) + AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + notification = FactoryBot.create(:notification, user: user, event: 'general') + + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + NotificationEmailJob.new.perform(notification.id) + end + assert_empty AdditionalNotificationEmailDeliveryJob.jobs + + AdditionalNotificationEmailService.remove(user: user) + notification = FactoryBot.create(:notification, user: user, event: 'general') + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + NotificationEmailJob.new.perform(notification.id) + end + assert_empty AdditionalNotificationEmailDeliveryJob.jobs + end + + def test_an_additional_copy_failure_does_not_fail_the_primary_delivery + user = FactoryBot.create(:user) + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create(:notification, user: user, event: 'general') + failing_delivery = Class.new do + def deliver_now + raise 'secondary smtp rejection' + end + end.new + + NotificationEmailJob.new.perform(notification.id) + copy_job = AdditionalNotificationEmailDeliveryJob.jobs.last + + NotificationsMailer.stub(:additional_notification_copy, ->(_notification, _address) { failing_delivery }) do + assert_raises(RuntimeError) do + AdditionalNotificationEmailDeliveryJob.new.perform(*copy_job['args']) + end + end + + assert_equal [user.email], ActionMailer::Base.deliveries.last.to + assert_equal 1, + user.additional_notification_email_audits.where(event: 'notification_copy_failed').count + end + + def test_audit_store_failure_does_not_retry_an_accepted_optional_copy + user = FactoryBot.create(:user) + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create(:notification, user: user, event: 'general') + NotificationEmailJob.new.perform(notification.id) + copy_job = AdditionalNotificationEmailDeliveryJob.jobs.last + + AdditionalNotificationEmailService.stub(:audit!, ->(*) { raise 'audit unavailable' }) do + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + AdditionalNotificationEmailDeliveryJob.new.perform(*copy_job['args']) + end + end + end + + def test_queue_and_audit_failure_do_not_retry_an_accepted_primary_message + user = FactoryBot.create(:user) + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create(:notification, user: user, event: 'general') + + AdditionalNotificationEmailDeliveryJob.stub(:perform_async, ->(*) { raise 'queue unavailable' }) do + AdditionalNotificationEmailService.stub(:audit!, ->(*) { raise 'audit unavailable' }) do + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + NotificationEmailJob.new.perform(notification.id) + end + end + end + end + + def test_category_opt_out_suppresses_both_destinations + user = FactoryBot.create(:user, receive_feedback_notifications: true) + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create( + :notification, + :feedback, + user: user, + event: 'task_comment_created' + ) + user.update!(receive_feedback_notifications: false) + + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + NotificationEmailJob.new.perform(notification.id) + end + assert_empty AdditionalNotificationEmailDeliveryJob.jobs + end + + def test_category_opt_out_after_primary_delivery_suppresses_the_optional_copy + user = FactoryBot.create(:user, receive_feedback_notifications: true) + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'secondary@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create( + :notification, + :feedback, + user: user, + event: 'task_comment_created' + ) + + NotificationEmailJob.new.perform(notification.id) + copy_job = AdditionalNotificationEmailDeliveryJob.jobs.last + user.update!(receive_feedback_notifications: false) + + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + AdditionalNotificationEmailDeliveryJob.new.perform(*copy_job['args']) + end + end + + def test_an_optional_address_lookup_failure_does_not_retry_the_primary_message + user = FactoryBot.create(:user) + notification = FactoryBot.create(:notification, user: user, event: 'general') + failing_lookup = -> { raise ActiveRecord::ConnectionNotEstablished, 'database unavailable' } + + user.stub(:additional_notification_email, failing_lookup) do + notification.stub(:user, user) do + Notification.stub(:find_by, notification) do + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + NotificationEmailJob.new.perform(notification.id) + end + end + end + end + + assert_empty AdditionalNotificationEmailDeliveryJob.jobs + assert_equal 1, user.additional_notification_email_audits.where(event: 'notification_copy_failed').count + end + + def test_a_rejected_copy_is_retried_without_the_address_in_the_error + user = FactoryBot.create(:user) + additional = AdditionalNotificationEmailService.request(user: user, email: 'secondary@example.org') + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create(:notification, user: user, event: 'general') + NotificationEmailJob.new.perform(notification.id) + copy_job = AdditionalNotificationEmailDeliveryJob.jobs.last + smtp_rejection = Class.new do + def deliver_now + raise '550 5.1.1 <secondary@example.org>: Recipient address rejected' + end + end.new + + error = NotificationsMailer.stub(:additional_notification_copy, ->(_notification, _address) { smtp_rejection }) do + assert_raises(AdditionalNotificationEmailService::DeliveryFailed) do + AdditionalNotificationEmailDeliveryJob.new.perform(*copy_job['args']) + end + end + + assert_not_includes error.message, 'secondary@example.org' + assert_nil error.cause + end + + def test_a_copy_for_a_removed_address_is_dropped_without_a_failure_audit + user = FactoryBot.create(:user) + additional = AdditionalNotificationEmailService.request(user: user, email: 'secondary@example.org') + AdditionalNotificationEmailService.verify(token: additional.verification_token) + notification = FactoryBot.create(:notification, user: user, event: 'general') + NotificationEmailJob.new.perform(notification.id) + copy_job = AdditionalNotificationEmailDeliveryJob.jobs.last + + AdditionalNotificationEmailService.remove(user: user) + + assert_no_difference -> { ActionMailer::Base.deliveries.count } do + AdditionalNotificationEmailDeliveryJob.new.perform(*copy_job['args']) + end + assert_equal 0, user.additional_notification_email_audits.where(event: 'notification_copy_failed').count + end + + def test_runtime_duplicate_suppression_prevents_two_messages + user = FactoryBot.create(:user, email: 'same@example.edu') + additional = AdditionalNotificationEmailService.request( + user: user, + email: 'different@example.org' + ) + AdditionalNotificationEmailService.verify(token: additional.verification_token) + # Simulate a legacy duplicate that predates the validation guard. + additional.update_column(:email, 'SAME@example.edu') # rubocop:disable Rails/SkipsModelValidations + notification = FactoryBot.create(:notification, user: user, event: 'general') + + assert_difference -> { ActionMailer::Base.deliveries.count }, 1 do + NotificationEmailJob.new.perform(notification.id) + end + assert_empty AdditionalNotificationEmailDeliveryJob.jobs + end +end diff --git a/test/sidekiq/push_notification_delivery_job_test.rb b/test/sidekiq/push_notification_delivery_job_test.rb new file mode 100644 index 0000000000..1c08137c70 --- /dev/null +++ b/test/sidekiq/push_notification_delivery_job_test.rb @@ -0,0 +1,130 @@ +# frozen_string_literal: true + +require 'test_helper' +require 'minitest/mock' + +class PushNotificationDeliveryJobTest < ActiveSupport::TestCase + VAPID_PUBLIC = 'BOs-KbIoHK7gUIX3i2_uEuDoouj-GKxB-mY9CRmLNmd4Wn-SSl254E1g6jR1ukL3e37p8uCpaMjOvfAB0BwzvSI=' + VAPID_PRIVATE = '_NFIWSUTdCdLJJFh87pf4ekQLmNYqsweZ4288NpVZaY=' + ENDPOINT = 'https://fcm.googleapis.com/fcm/send/job-retry-browser' + + setup do + PushNotificationDeliveryJob.clear + end + + def test_perform_delivers_the_reloaded_notification + notification = FactoryBot.create(:notification, event: 'general') + delivered = nil + + PushNotificationService.stub(:deliver, ->(record) { delivered = record }) do + PushNotificationDeliveryJob.new.perform(notification.id) + end + + assert_equal notification, delivered + end + + def test_missing_notification_is_raised_so_a_pre_commit_race_is_retried + PushNotificationService.stub(:deliver, ->(_record) { flunk 'missing row must not be delivered' }) do + assert_raises(ActiveRecord::RecordNotFound) do + PushNotificationDeliveryJob.new.perform(-1) + end + end + end + + def test_delivery_failure_is_raised_so_sidekiq_can_retry + notification = FactoryBot.create(:notification, event: 'general') + failure = ->(_record) { raise 'push provider unavailable' } + + PushNotificationService.stub(:deliver, failure) do + error = assert_raises(RuntimeError) do + PushNotificationDeliveryJob.new.perform(notification.id) + end + assert_equal 'push provider unavailable', error.message + end + end + + def test_no_delivery_when_the_preference_was_turned_off_after_queueing + user = FactoryBot.create(:user, receive_feedback_notifications: true) + notification = FactoryBot.create( + :notification, + :feedback, + user: user, + event: 'task_comment_created', + message: 'Queued while the category was still on.' + ) + + # retry: 3 means the job can run well after it was queued. + user.update!(receive_feedback_notifications: false) + + PushNotificationService.stub(:deliver, ->(_record) { flunk 'push must not be delivered after the category is off' }) do + PushNotificationDeliveryJob.new.perform(notification.id) + end + end + + def test_a_type_without_a_preference_is_still_delivered + user = FactoryBot.create( + :user, + receive_task_notifications: false, + receive_feedback_notifications: false, + receive_portfolio_notifications: false + ) + notification = FactoryBot.create( + :notification, + user: user, + event: 'general', + message: 'General notices ignore the category toggles.' + ) + delivered = nil + + PushNotificationService.stub(:deliver, ->(record) { delivered = record }) do + PushNotificationDeliveryJob.new.perform(notification.id) + end + + assert_equal notification, delivered + end + + def test_real_provider_failure_reaches_the_sidekiq_retry_boundary + notification = FactoryBot.create(:notification, event: 'general') + FactoryBot.create( + :push_subscription, + user: notification.user, + endpoint: ENDPOINT + ) + stub_request(:post, ENDPOINT).to_return(status: 503) + + with_vapid_keys do + assert_raises(PushNotificationService::DeliveryError) do + PushNotificationDeliveryJob.new.perform(notification.id) + end + end + end + + def test_async_payload_contains_only_the_notification_id + notification = FactoryBot.create(:notification, event: 'general') + jid = PushNotificationDeliveryJob.perform_async(notification.id) + job = PushNotificationDeliveryJob.jobs.find do |candidate| + candidate['jid'] == jid + end + + assert_not_nil job + assert_equal 'PushNotificationDeliveryJob', job['class'] + assert_equal 'notifications', job['queue'] + assert_equal [notification.id], job['args'] + assert_equal 'notifications', PushNotificationDeliveryJob.get_sidekiq_options['queue'].to_s + assert_equal 3, PushNotificationDeliveryJob.get_sidekiq_options['retry'] + end + + private + + def with_vapid_keys + names = %w[DOUBTFIRE_VAPID_PUBLIC_KEY DOUBTFIRE_VAPID_PRIVATE_KEY] + previous = names.index_with { |name| ENV.fetch(name, nil) } + ENV['DOUBTFIRE_VAPID_PUBLIC_KEY'] = VAPID_PUBLIC + ENV['DOUBTFIRE_VAPID_PRIVATE_KEY'] = VAPID_PRIVATE + yield + ensure + previous.each do |name, value| + value.nil? ? ENV.delete(name) : ENV[name] = value + end + end +end diff --git a/test/sidekiq/scheduled_job_test.rb b/test/sidekiq/scheduled_job_test.rb index e21285fdf3..6940c6028c 100644 --- a/test/sidekiq/scheduled_job_test.rb +++ b/test/sidekiq/scheduled_job_test.rb @@ -1,21 +1,42 @@ # frozen_string_literal: true require 'test_helper' -class TiiCheckProgressJobTest < ActiveSupport::TestCase +require 'sidekiq_unique_jobs/testing' +class TiiCheckProgressJobTest < ActiveSupport::TestCase def test_jobs_are_scheduled + # Clear fake jobs and any unique-job locks left by an earlier test run. + Sidekiq::Job.clear_all Sidekiq::Cron::Job.destroy_all! - Sidekiq::Cron::Job.load_from_hash!(YAML.load_file(Rails.root.join('config/schedule.yml'))) - assert_equal 6, Sidekiq::Cron::Job.all.count, Sidekiq::Cron::Job.all.map(&:name) + Sidekiq::Cron::Job.load_from_hash!( + YAML.load_file(Rails.root.join('config/schedule.yml')) + ) + + jobs = Sidekiq::Cron::Job.all + peer_progress_job = + jobs.find { |job| job.name == 'aggregate_peer_progress' } + + assert_equal 15, jobs.count, jobs.map(&:name) + assert_not_nil peer_progress_job + assert_equal 'AggregatePeerProgressJob', peer_progress_job.klass + + # Sidekiq::Cron::Job.all returns an Array, not an ActiveRecord relation. + jobs.each(&:enqueue!) - Sidekiq::Cron::Job.all.each(&:enqueue!) assert_equal 1, TiiRegisterWebHookJob.jobs.count assert_equal 1, TiiCheckProgressJob.jobs.count assert_equal 1, ClearAccessTokensJob.jobs.count assert_equal 1, RefreshModerationFeedbackTimestampsJob.jobs.count + assert_equal 1, AggregatePeerProgressJob.jobs.count assert_equal 1, AggregateTaskCompletionStatsJob.jobs.count assert_equal 1, PollCommunicationSetSchedulesJob.jobs.count + assert_equal 1, SendNewTaskAvailableNotificationsJob.jobs.count + assert_equal 1, SendDueSoonRemindersJob.jobs.count + assert_equal 1, CheckUnitSimilarityJob.jobs.count + assert_equal 1, SyncTeamsAnnouncementsJob.jobs.count + assert_equal 1, SendUnitSessionRemindersJob.jobs.count + # One student digest run per cadence a student can choose. + assert_equal %w[daily monthly weekly], SendDigestEmailsJob.jobs.map { |job| job['args'].first }.sort # assert_equal 1, ArchiveOldUnitsJob.jobs.count end - end diff --git a/test/sidekiq/send_due_soon_reminders_job_test.rb b/test/sidekiq/send_due_soon_reminders_job_test.rb new file mode 100644 index 0000000000..86f40a3c36 --- /dev/null +++ b/test/sidekiq/send_due_soon_reminders_job_test.rb @@ -0,0 +1,263 @@ +# frozen_string_literal: true + +require 'test_helper' +# test_helper does not pull this in, and Object#stub comes from it. +require 'minitest/mock' + +class SendDueSoonRemindersJobTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + EVENT = 'task_due_soon' + WINDOW_DAYS = SendDueSoonRemindersJob::WINDOW_DAYS + + setup do + @unit = FactoryBot.create(:unit, task_count: 0) + + # This job sweeps every active unit there is, which is the whole point of + # it. rake db:populate seeds four more, each with a cohort and task + # definitions of its own, and their students would then land in every count + # in this file: the first run of these tests expected 11 notifications and + # got 27. Narrowing the world to the unit under test is what makes a plain + # Notification.count assertion mean what it says. + Unit.where.not(id: @unit.id).update_all(active: false) + + @task_def = FactoryBot.create( + :task_definition, + unit: @unit, + target_grade: 0, + start_date: Time.zone.now - 1.week, + target_date: Time.zone.now + 2.days + ) + + ActionMailer::Base.deliveries.clear + end + + # The students who most need a reminder are the ones who have not opened the + # task, and OnTrack has no Task row for them until somebody touches it. A + # sweep that read Task rows would miss exactly those people, and one that + # called task_for_task_definition would silently create a row per student per + # task every morning. + def test_reminds_every_eligible_student_without_creating_tasks + expected = @unit.active_projects.count + + assert_operator expected, :>=, 2 + assert_equal 0, @task_def.tasks.count + + assert_difference 'Notification.count', expected do + assert_no_difference 'Task.count' do + run_job + end + end + + assert_equal expected, ActionMailer::Base.deliveries.count + end + + def test_does_not_remind_about_a_deadline_further_out + @task_def.update!(target_date: Time.zone.now + (WINDOW_DAYS + 1).days) + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_reminds_on_the_last_day_of_the_window + @task_def.update!(target_date: Time.zone.now + WINDOW_DAYS.days) + + # The far edge, where an off by one turns the window into WINDOW_DAYS - 1 + # without anything else looking wrong. + assert_difference 'Notification.count', @unit.active_projects.count do + run_job + end + end + + def test_reminds_about_something_due_today + @task_def.update!(target_date: Time.zone.now) + + assert_difference 'Notification.count', @unit.active_projects.count do + run_job + end + end + + def test_does_not_remind_once_the_deadline_has_passed + @task_def.update!(target_date: Time.zone.now - 1.day) + + # Overdue is a different message and a different ticket. A reminder saying + # a task is due soon when it is already late is worse than saying nothing. + assert_no_difference 'Notification.count' do + run_job + end + end + + # This is the whole reason the job carries a duplicate guard. It runs every + # morning and the task is still due soon tomorrow morning. + def test_does_not_remind_the_same_student_twice + run_job + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_does_not_remind_a_withdrawn_student + project = @unit.projects.find_by!(enrolled: false) + project.update!(target_grade: 3) + + run_job + + assert_not Notification.exists?(user: project.student, event: EVENT) + end + + def test_does_not_remind_for_an_inactive_unit + @unit.update!(active: false) + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_respects_task_notification_preference + project = @unit.active_projects.first + project.student.update!(receive_task_notifications: false) + + run_job + + assert_not Notification.exists?(user: project.student, event: EVENT) + end + + def test_does_not_remind_a_student_the_task_is_not_assigned_to + @task_def.update!(target_grade: 3) + below = @unit.active_projects.where('projects.target_grade < 3') + + assert_operator below.count, :>, 0 + + run_job + + below.each do |project| + assert_not Notification.exists?(user: project.student, event: EVENT) + end + end + + # An extension moves the deadline for one student and nobody else, so the + # date has to be read per student rather than off the task definition. + def test_uses_the_students_own_extended_deadline + project = @unit.active_projects.first + project.task_for_task_definition(@task_def).update!(extensions: 1) + + run_job + + assert_not Notification.exists?(user: project.student, event: EVENT) + + # Everybody else is still on the original date and still gets one, so this + # is the extension being read and not the whole sweep falling over. + assert Notification.exists?(user: @unit.active_projects.second.student, event: EVENT) + end + + # :discuss and :demonstrate mean the student has submitted and is waiting on a + # tutor, so a reminder is both wrong and the kind of wrong that teaches people + # to ignore notifications. + def test_does_not_remind_about_a_task_that_is_waiting_on_a_tutor + project = @unit.active_projects.first + project.task_for_task_definition(@task_def).update!(task_status: TaskStatus.discuss) + + run_job + + assert_not Notification.exists?(user: project.student, event: EVENT) + end + + # A unit with flexible dates gives each target grade its own deadline, and + # that override applies before any Task row exists. Falling back to the task + # definition's own target date for a student with no row is wrong by however + # far apart those two dates are, and it is wrong in both directions: silence + # when something is due in two days, or a reminder a week early. + def test_uses_the_grade_deadline_when_the_unit_has_flexible_dates + @unit.update!(allow_flexible_dates: true) + @task_def.update!(target_date: Time.zone.now + 10.days) + + project = @unit.active_projects.find_by!(target_grade: 2) + @task_def.grade_due_dates.create!( + target_grade: 2, + start_date: Time.zone.now - 1.week, + target_due_date: Time.zone.now + 2.days + ) + + assert_equal 0, @task_def.tasks.count + + run_job + + assert Notification.exists?(user: project.student, event: EVENT) + + # Nobody else moved, so this is the override being read rather than the + # whole window sliding. + other = @unit.active_projects.where.not(target_grade: 2).first + + assert_not Notification.exists?(user: other.student, event: EVENT) + end + + # Logging a failure and carrying on leaves perform successful, Sidekiq + # schedules no retry, and a student whose task is due today is filtered out as + # overdue tomorrow. That reminder is then gone for good. + def test_a_failure_is_raised_so_sidekiq_retries + raising = ->(**_args) { raise 'notification failed' } + + NotificationService.stub(:notify, raising) do + assert_raises(RuntimeError) { run_job } + end + end + + def test_message_and_link_are_privacy_safe + project = @unit.active_projects.first + + run_job + + notification = Notification.find_by!(user: project.student, event: EVENT) + + assert_includes notification.message, @task_def.abbreviation + assert_includes notification.message, @unit.code + + # The date stays out, the same as task_due_date_changed. The row outlives + # the deadline it describes, so "due on the 14th" is wrong a week later + # while "due soon" only ever stops being interesting. + assert_not_includes notification.message, @task_def.target_date.to_date.to_s + assert_not_includes notification.message, @task_def.target_date.to_date.iso8601 + + assert_equal( + "/projects/#{project.id}/dashboard/#{@task_def.abbreviation}", + notification.link + ) + + assert_valid_push_payload( + notification, + expected_link: "/projects/#{project.id}/dashboard/#{@task_def.abbreviation}" + ) + end + + def test_event_specific_template_is_used + run_job + + assert_includes delivered_body, 'The deadline is not included in this email' + end + + def test_the_schedule_entry_points_at_this_job + schedule = YAML.load_file(Rails.root.join('config/schedule.yml')) + + assert_equal( + 'SendDueSoonRemindersJob', + schedule.dig('send_due_soon_reminders', 'class') + ) + end + + private + + def run_job + SendDueSoonRemindersJob.new.perform + NotificationEmailJob.drain + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + return mail.body.decoded unless mail.multipart? + + mail.parts.map { |part| part.body.decoded }.join("\n") + end +end diff --git a/test/sidekiq/send_new_task_available_notifications_job_test.rb b/test/sidekiq/send_new_task_available_notifications_job_test.rb new file mode 100644 index 0000000000..fd64d4c0f0 --- /dev/null +++ b/test/sidekiq/send_new_task_available_notifications_job_test.rb @@ -0,0 +1,339 @@ +# frozen_string_literal: true + +require 'test_helper' +require 'minitest/mock' +require 'tempfile' + +class SendNewTaskAvailableNotificationsJobTest < ActiveSupport::TestCase + include ActiveSupport::Testing::TimeHelpers + + setup do + @unit = FactoryBot.create( + :unit, + with_students: false, + task_count: 0, + tutorials: 0, + outcome_count: 0, + staff_count: 0, + campus_count: 1, + active: true + ) + Unit.where.not(id: @unit.id).update_all(active: false) + + @student = FactoryBot.create( + :user, + :student, + receive_task_notifications: true + ) + @project = FactoryBot.create( + :project, + unit: @unit, + campus: Campus.first, + user: @student, + enrolled: true, + target_grade: 2 + ) + @release_date = 2.days.from_now.beginning_of_day + @task_definition = FactoryBot.create( + :task_definition, + unit: @unit, + outcome_count: 0, + target_grade: 1, + start_date: @release_date, + target_date: @release_date + 1.week + ) + @task_definition.enable_new_task_notifications! + end + + def test_notifies_on_release_date_without_creating_task_rows + assert_no_difference 'Notification.count' do + run_job + end + + travel_to @release_date.noon do + assert_difference 'Notification.count', 1 do + assert_no_difference 'Task.count' do + run_job + end + end + + assert_no_difference 'Notification.count' do + run_job + end + end + end + + def test_uses_grade_specific_start_date + @unit.update!(allow_flexible_dates: true) + @task_definition.update!(start_date: @release_date + 1.week) + @task_definition.grade_due_dates.create!( + target_grade: @project.target_grade, + start_date: @release_date, + target_due_date: @release_date + 1.week + ) + + travel_to @release_date.noon do + assert_difference 'Notification.count', 1 do + assert_no_difference 'Task.count' do + run_job + end + end + end + end + + def test_skips_a_student_below_the_task_target_grade + @project.update!(target_grade: 0) + + travel_to @release_date.noon do + assert_no_difference 'Notification.count' do + run_job + end + end + end + + def test_uses_student_specific_start_date + @unit.update!(allow_flexible_dates: true) + @task_definition.update!(start_date: @release_date + 1.week) + @project.task_for_task_definition(@task_definition).update!( + target_start_date: @release_date + ) + + travel_to @release_date.noon do + assert_difference 'Notification.count', 1 do + assert_no_difference 'Task.count' do + run_job + end + end + end + end + + def test_catches_up_after_a_missed_release_day + @task_definition.update_column(:created_at, 1.month.ago) + + travel_to (@release_date + 1.day).noon do + assert_difference 'Notification.count', 1 do + run_job + end + end + end + + def test_direct_job_repairs_a_failed_tracking_write_for_future_delivery + @task_definition.update_column(:new_task_notifications_from, nil) + marker_failure = -> { raise StandardError, 'temporary marker failure' } + + assert_difference -> { NewTaskAvailableNotificationJob.jobs.size }, 1 do + @task_definition.stub(:enable_new_task_notifications!, marker_failure) do + NewTaskAvailableNotificationJob.track_and_enqueue(@task_definition) + end + end + + assert_nil @task_definition.reload.new_task_notifications_from + assert_no_difference 'Notification.count' do + NewTaskAvailableNotificationJob.new.perform(@task_definition.id) + end + assert_not_nil @task_definition.reload.new_task_notifications_from + + travel_to @release_date.noon do + assert_difference 'Notification.count', 1 do + run_job + end + end + end + + def test_future_csv_import_is_tracked_and_notified_on_release + @task_definition.update_column(:new_task_notifications_from, nil) + source_unit = FactoryBot.create( + :unit, + with_students: false, + task_count: 0, + tutorials: 0, + outcome_count: 0, + staff_count: 0, + campus_count: 0, + active: false, + start_date: @unit.start_date, + end_date: @unit.end_date + ) + abbreviation = "CSV#{SecureRandom.hex(3)}" + FactoryBot.create( + :task_definition, + unit: source_unit, + outcome_count: 0, + abbreviation: abbreviation, + target_grade: 1, + start_date: @release_date, + target_date: @release_date + 1.week + ) + csv = Tempfile.new(['future-task', '.csv']) + csv.write(source_unit.task_definitions_csv) + csv.rewind + + result = @unit.import_tasks_from_csv(csv) + assert_empty result[:errors], result.inspect + + imported = @unit.task_definitions.find_by!(abbreviation: abbreviation) + assert_not_nil imported.new_task_notifications_from + notification_count = lambda do + Notification.where( + user: @student, + event: 'new_task_available', + link: "/projects/#{@project.id}/dashboard/#{imported.abbreviation}" + ).count + end + + travel_to (@release_date - 1.day).noon do + assert_operator imported.start_date.to_date, :>, Time.zone.today + assert_no_difference notification_count do + assert_no_difference 'Task.count' do + NewTaskAvailableNotificationJob.new.perform(imported.id) + end + end + end + + travel_to @release_date.noon do + assert_difference notification_count, 1 do + assert_no_difference 'Task.count' do + run_job + end + end + end + ensure + csv&.close! + source_unit&.destroy + end + + def test_does_not_backfill_a_historical_task + @task_definition.update_columns( + created_at: 1.month.ago, + start_date: Time.zone.today - 1.day, + new_task_notifications_from: Time.current + ) + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_does_not_backfill_a_recent_task_created_before_tracking_started + @task_definition.update_columns( + created_at: 1.day.ago, + start_date: 1.month.ago, + target_date: 3.weeks.ago, + new_task_notifications_from: Time.current + ) + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_accepts_a_rolling_writer_marker_just_after_creation + @task_definition.update_columns( + created_at: 2.days.ago, + start_date: 3.days.ago, + target_date: 1.week.from_now, + new_task_notifications_from: 2.days.ago + 10.seconds + ) + + assert_difference 'Notification.count', 1 do + run_job + end + end + + def test_ignores_inactive_units_and_withdrawn_projects + travel_to @release_date.noon do + @unit.update!(active: false) + + assert_no_difference 'Notification.count' do + run_job + end + + @unit.update!(active: true) + @project.update!(enrolled: false) + + assert_no_difference 'Notification.count' do + run_job + end + end + end + + def test_catches_up_after_an_inactive_unit_is_reactivated + @unit.update!(active: false) + + travel_to @release_date.noon do + assert_no_difference 'Notification.count' do + run_job + end + end + + @unit.update!(active: true) + travel_to (@release_date + 1.day).noon do + assert_difference 'Notification.count', 1 do + run_job + end + end + end + + def test_ignores_definitions_not_created_by_a_supported_workflow + unsupported = FactoryBot.create( + :task_definition, + unit: @unit, + outcome_count: 0, + target_grade: 1, + start_date: @release_date, + target_date: @release_date + 1.week + ) + assert_nil unsupported.reload.new_task_notifications_from + @task_definition.update_column(:new_task_notifications_from, nil) + + travel_to @release_date.noon do + assert_no_difference 'Notification.count' do + run_job + end + end + end + + def test_rollover_notifies_students_enrolled_before_the_copied_task_releases + rolled_unit = @unit.rollover( + nil, + Time.zone.today + 4.weeks, + Time.zone.today + 16.weeks, + "ROLLED-#{SecureRandom.hex(3)}" + ) + rolled_task = rolled_unit.task_definitions.find_by!( + abbreviation: @task_definition.abbreviation + ) + rolled_project = FactoryBot.create( + :project, + unit: rolled_unit, + campus: Campus.first, + user: @student, + enrolled: true, + target_grade: @project.target_grade + ) + @unit.update!(active: false) + + travel_to rolled_task.start_date.to_date.noon do + assert_difference 'Notification.count', 1 do + run_job + end + + notification = Notification.find_by!( + user: @student, + event: NewTaskAvailableNotificationJob::EVENT + ) + assert_equal( + "/projects/#{rolled_project.id}/dashboard/#{rolled_task.abbreviation}", + notification.link + ) + end + ensure + rolled_unit&.destroy + end + + private + + def run_job + SendNewTaskAvailableNotificationsJob.new.perform + end +end diff --git a/test/sidekiq/task_due_date_changed_notification_job_test.rb b/test/sidekiq/task_due_date_changed_notification_job_test.rb new file mode 100644 index 0000000000..f6ea2340a7 --- /dev/null +++ b/test/sidekiq/task_due_date_changed_notification_job_test.rb @@ -0,0 +1,293 @@ +# frozen_string_literal: true + +require 'test_helper' + +class TaskDueDateChangedNotificationJobTest < ActiveSupport::TestCase + include TestHelpers::PushNotificationHelper + + EVENT = 'task_due_date_changed' + + setup do + @unit = FactoryBot.create(:unit, task_count: 0) + @task_def = FactoryBot.create( + :task_definition, + unit: @unit, + target_grade: 1 + ) + + @previous_due_date = @task_def[:due_date]&.to_date&.iso8601 + changed_due_date = (@task_def.due_date + 1.week).to_date + + @task_def.update!(due_date: changed_due_date) + @new_due_date = changed_due_date.iso8601 + + ActionMailer::Base.deliveries.clear + end + + def test_cohort_above_the_limit_needs_explicit_override + old_limit = ENV.fetch('DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT', nil) + ENV['DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT'] = '1' + assert_no_difference 'Notification.count' do + TaskDueDateChangedNotificationJob.new.perform(@task_def.id, @previous_due_date, @new_due_date, 'bounded') + end + assert_difference 'Notification.count', eligible_projects.count do + TaskDueDateChangedNotificationJob.new.perform(@task_def.id, @previous_due_date, @new_due_date, 'bounded', true) + end + ensure + old_limit.nil? ? ENV.delete('DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT') : ENV['DOUBTFIRE_NOTIFICATION_FANOUT_LIMIT'] = old_limit + end + + def test_notifies_every_eligible_student_without_creating_tasks + expected = eligible_projects.count + + assert_operator expected, :>=, 2 + assert_equal 0, @task_def.tasks.count + + assert_difference 'Notification.count', expected do + assert_no_difference 'Task.count' do + run_job + end + end + + assert_equal expected, ActionMailer::Base.deliveries.count + end + + def test_does_not_notify_student_below_target_grade + project = @unit.active_projects.find_by!(target_grade: 0) + + run_job + + assert_not Notification.exists?( + user: project.student, + event: EVENT + ) + end + + def test_does_not_notify_withdrawn_student + project = @unit.projects.find_by!(enrolled: false) + project.update!(target_grade: 3) + + run_job + + assert_not Notification.exists?( + user: project.student, + event: EVENT + ) + end + + def test_respects_task_notification_preference + project = eligible_projects.first + project.student.update!(receive_task_notifications: false) + + run_job + + assert_not Notification.exists?( + user: project.student, + event: EVENT + ) + end + + def test_does_not_notify_for_inactive_unit + @unit.update!(active: false) + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_skips_stale_job_after_another_due_date_change + @task_def.update!(due_date: @task_def.due_date + 1.week) + + assert_no_difference 'Notification.count' do + run_job + end + end + + def test_direct_model_change_does_not_enqueue_job + task_definition = FactoryBot.create( + :task_definition, + unit: @unit, + target_grade: 1 + ) + + assert_no_difference( + -> { TaskDueDateChangedNotificationJob.jobs.size } + ) do + task_definition.update!( + due_date: task_definition.due_date + 1.day + ) + end + end + + def test_message_and_link_are_privacy_safe + project = eligible_projects.first + + run_job + + notification = Notification.find_by!( + user: project.student, + event: EVENT + ) + + assert_includes notification.message, @task_def.abbreviation + assert_includes notification.message, @unit.code + assert_not_includes notification.message, @new_due_date + + assert_equal( + "/projects/#{project.id}/dashboard/#{@task_def.abbreviation}", + notification.link + ) + push = assert_valid_push_payload( + notification, + expected_link: "/projects/#{project.id}/dashboard/#{@task_def.abbreviation}" + ) + assert_not_includes push['body'], @new_due_date + end + + def test_event_specific_template_is_used + run_job + + assert_includes( + delivered_body, + 'The new due date is not included in this email' + ) + end + + # Regression: a task abbreviation with a space must be percent-encoded in the + # stored link, so the email href and plain-text URL are valid. Every other + # notification event encodes the abbreviation; this job was the one that did + # not, so a space landed raw in the href. + def test_link_is_url_encoded_for_a_spaced_abbreviation + @task_def.update!(abbreviation: 'AB 1.1') + project = eligible_projects.first + + run_job + + notification = Notification.find_by!( + user: project.student, + event: EVENT + ) + + assert_equal( + "/projects/#{project.id}/dashboard/AB%201.1", + notification.link + ) + end + + # Regression: re-running the sweep for the same change must not notify a + # student twice. retry: 3 plus a duplicate enqueue can run perform again, so + # the notify carries a dedupe_key; the second run finds the existing row + # instead of sending a second email. + def test_re_running_the_same_change_does_not_notify_twice + expected = eligible_projects.count + assert_operator expected, :>=, 2 + + assert_difference 'Notification.count', expected do + run_job + end + + assert_no_difference 'Notification.count' do + assert_no_difference(-> { ActionMailer::Base.deliveries.count }) do + run_job + end + end + end + + # Regression: a genuine second change to a different date must notify again. + # The dedupe_key includes the new due date, so a new date is a new key. + def test_a_later_change_to_a_different_date_notifies_again + expected = eligible_projects.count + + assert_difference 'Notification.count', expected do + run_job + end + + later = (@task_def.due_date + 2.weeks).to_date + @task_def.update!(due_date: later) + @new_due_date = later.iso8601 + + assert_difference 'Notification.count', expected do + run_job + end + end + + # Regression: a transient failure for one recipient must re-raise so Sidekiq + # retries the sweep, not be logged and swallowed. This event fires once off a + # convenor's edit and is never swept for again, so a swallowed failure loses + # that student's notification for good. Matches NewTaskAvailableNotificationJob. + def test_re_raises_so_sidekiq_retries_when_a_recipient_fails + NotificationService.stub(:notify, ->(*_args, **_kwargs) { raise 'transient insert failure' }) do + assert_raises(RuntimeError) do + TaskDueDateChangedNotificationJob.new.perform( + @task_def.id, + @previous_due_date, + @new_due_date + ) + end + end + end + + def test_returning_to_a_previously_used_date_notifies_again + expected = eligible_projects.count + first_date = @new_due_date + run_job + + @task_def.update!(due_date: @task_def.due_date + 2.weeks) + @new_due_date = @task_def[:due_date].to_date.iso8601 + run_job + + @task_def.update!(due_date: first_date) + @new_due_date = first_date + assert_difference 'Notification.count', expected do + run_job + end + end + + def test_explicit_occurrence_is_stable_across_retries_and_unrelated_edits + job = TaskDueDateChangedNotificationJob.new + change_id = SecureRandom.uuid + job.perform(@task_def.id, @previous_due_date, @new_due_date, change_id) + @task_def.update!(name: 'Updated task name') + + assert_no_difference 'Notification.count' do + job.perform(@task_def.id, @previous_due_date, @new_due_date, change_id) + end + end + + def test_legacy_three_argument_job_uses_its_stable_sidekiq_id + job = TaskDueDateChangedNotificationJob.new + job.jid = 'legacy-job-id' + job.perform(@task_def.id, @previous_due_date, @new_due_date) + @task_def.update!(name: 'Updated task name') + + assert_no_difference 'Notification.count' do + job.perform(@task_def.id, @previous_due_date, @new_due_date) + end + end + + private + + def run_job + TaskDueDateChangedNotificationJob.new.perform( + @task_def.id, + @previous_due_date, + @new_due_date + ) + NotificationEmailJob.drain + end + + def eligible_projects + @unit.active_projects.where( + 'projects.target_grade >= ?', + @task_def.target_grade + ) + end + + def delivered_body + mail = ActionMailer::Base.deliveries.last + return '' if mail.nil? + return mail.body.decoded unless mail.multipart? + + mail.parts.map { |part| part.body.decoded }.join("\n") + end +end