Skip to main content

Jobs & Scheduling

PBS Knowledge uses a job queue system (BullMQ) to handle background tasks like data synchronization, email sending, and scheduled maintenance.

Overview​

What Are Jobs?​

Jobs are background tasks that:

  • Run asynchronously without blocking users
  • Process large amounts of data
  • Execute on schedules
  • Retry on failure

Job Types​

QueuePurpose
EMAIL_QUEUESending transactional emails
SYNC_QUEUEData synchronization tasks
PUBLICATION_SYNC_QUEUEPublication import/updates
BACKUP_QUEUEDatabase backups

Viewing Jobs​

Job Dashboard​

  1. Go to Admin → System → Jobs
  2. See job queues and status
  3. View recent job history
  4. Monitor active jobs

Job Status​

StatusMeaning
waitingQueued, not yet started
activeCurrently processing
completedSuccessfully finished
failedError occurred
delayedScheduled for later
pausedQueue is paused

Job Details​

Click any job to see:

  • Job ID and type
  • Input data
  • Start and end times
  • Duration
  • Error messages (if failed)
  • Retry attempts

Scheduled Jobs​

Current Schedules​

View scheduled jobs:

  1. Go to Admin → System → Schedulers
  2. See all scheduled tasks
  3. View next run times
  4. Check schedule patterns

Common Schedules​

JobDefault SchedulePurpose
Publication SyncDaily 2 AMImport new publications
Citation UpdateWeekly SundayUpdate citation counts
Dartmouth SyncDaily 3 AMSync people/courses
Database BackupDaily 4 AMBackup database
CleanupWeeklyRemove old job data

Cron Patterns​

Schedules use cron syntax:

┌────────────── minute (0-59)
│ ┌──────────── hour (0-23)
│ │ ┌────────── day of month (1-31)
│ │ │ ┌──────── month (1-12)
│ │ │ │ ┌────── day of week (0-6, Sunday=0)
│ │ │ │ │
* * * * *

Examples:
0 2 * * *     = Daily at 2:00 AM
0 0 * * 0     = Weekly on Sunday at midnight
0 */6 * * *   = Every 6 hours
30 8 1 * *    = Monthly on the 1st at 8:30 AM

Managing Schedulers​

Creating a Scheduler​

  1. Go to Admin → Schedulers
  2. Click Create Scheduler
  3. Configure:
    • Name
    • Job type
    • Cron pattern
    • Job data
    • Active status
  4. Save

Editing Schedules​

  1. Find the scheduler
  2. Click Edit
  3. Modify schedule or parameters
  4. Save changes

Pausing Schedules​

To temporarily stop:

  1. Find the scheduler
  2. Click Pause
  3. Jobs won't run until resumed
  4. Click Resume to restart

Manual Job Execution​

Running Jobs Manually​

Trigger a job immediately:

  1. Go to Admin → Jobs
  2. Find the job type
  3. Click Run Now
  4. Optionally provide parameters
  5. Monitor execution

Use Cases​

Manual runs are useful for:

  • Testing new configurations
  • Recovering from failures
  • Immediate data updates
  • Troubleshooting

Job Configuration​

Queue Settings​

Configure queue behavior:

  • Concurrency - Parallel job limit
  • Rate Limit - Jobs per time period
  • Retry Policy - Failure handling
  • Timeout - Maximum execution time

Retry Configuration​

{
  attempts: 3,           // Number of retries
  backoff: {
    type: 'exponential', // Backoff strategy
    delay: 1000          // Initial delay (ms)
  }
}

Priority​

Jobs can have priorities:

  • Higher priority runs first
  • Default priority is 0
  • Lower numbers = higher priority

Monitoring​

Health Indicators​

Monitor job system health:

  • Queue length (waiting jobs)
  • Processing rate
  • Failure rate
  • Average duration

Alerts​

Configure alerts for:

  • Queue backup (too many waiting)
  • High failure rate
  • Long-running jobs
  • System errors

Metrics Dashboard​

View metrics:

  1. Go to Admin → System → Metrics
  2. See job statistics
  3. View trends over time
  4. Identify bottlenecks

Troubleshooting​

Jobs Not Running​

  1. Check Redis connectivity
  2. Verify worker is running
  3. Check scheduler is active
  4. Review error logs

Jobs Failing​

  1. Check job error message
  2. Verify external service status
  3. Check rate limits
  4. Review input data

Queue Backup​

If jobs are piling up:

  1. Check worker capacity
  2. Increase concurrency if possible
  3. Pause non-critical jobs
  4. Address failed jobs blocking queue

Clearing Stuck Jobs​

To clear problematic jobs:

  1. Go to job queue
  2. Select stuck jobs
  3. Click Remove or Retry
  4. Monitor queue recovery

Best Practices​

Schedule Timing​

  • Run heavy jobs during off-hours
  • Stagger schedules to avoid conflicts
  • Allow buffer between dependent jobs

Error Handling​

  • Configure appropriate retry policies
  • Monitor failure rates
  • Set up alerting
  • Document recovery procedures

Performance​

  • Keep job payloads small
  • Use appropriate concurrency
  • Monitor queue lengths
  • Scale workers as needed