Heratio Help Center article. Category: User Guide.

Digital Preservation User Guide

Digital Preservation is a built-in Heratio capability, not a separate add-on. There is nothing to install. CLI tasks run through Heratio's php artisan console and the in-app Scheduler. For the wider integrity and authenticity picture, see the Content Authenticity in Heratio reference and the Integrity Assurance user guide.

Overview

Digital Preservation in Heratio helps you protect your digital collections by ensuring files remain unchanged over time, tracking all preservation activities, identifying file formats that may require attention, scanning for viruses, converting files to preservation-safe formats, and replicating backups to multiple locations.

Key Benefits:

• Integrity Assurance - Detect if files have been corrupted or modified
• Audit Trail - Complete history of all preservation activities
• Format Awareness - Know which files may need migration to safer formats
• Virus Protection - Scan files for malware using ClamAV
• Format Conversion - Convert files to archival-quality formats
• Backup Replication - Replicate files to multiple backup targets
• Compliance - Meet OAIS and PREMIS preservation standards

Getting Started

Accessing the Preservation Dashboard

1. Log in as an administrator
2. Navigate to Central Dashboard -> Digital Preservation
3. Or go directly to: /admin/preservation

Accessing Preservation Settings

1. Go to Admin -> AHG Settings
2. Click Preservation & Backup card
3. Or navigate to: /admin/preservation/policies

Understanding the Dashboard

Statistics Overview

The dashboard displays key metrics at a glance:

Core Features

1. Checksums

Checksums are digital fingerprints that uniquely identify file contents. If a file changes (even one bit), the checksum changes.

How Checksums Work

Supported Algorithms

Generating Checksums

Automatic: Checksums are generated when files are uploaded (if enabled in settings).

Manual:

1. Go to a digital object's page
2. Click Preservation tab
3. Click Generate Checksums

2. Fixity Verification

Fixity checks verify that files haven't changed since checksums were created.

Running Fixity Checks

Single Object:

1. Navigate to the digital object
2. Click Preservation -> Verify Fixity

Batch Verification (CLI):

# Verify up to 100 objects whose last check is older than 90 days (defaults)
php artisan ahg:preservation-fixity

# Verify objects not checked in 7+ days, up to 500
php artisan ahg:preservation-fixity --age=7 --limit=500

# Use SHA-512 instead of the SHA-256 default
php artisan ahg:preservation-fixity --algorithm=sha512 --limit=200

# Verify a single digital object
php artisan ahg:preservation-fixity --digital-object-id=123

# Print a summary report and exit
php artisan ahg:preservation-fixity --report

Restoring from backup after a failure:

When fixity detects corruption or a missing file, restore the affected object from a verified backup copy. Backups are managed by the Backup & Replication module - replicas are independently checksummed (see Backup Replication). Use the backup restore commands to recover a specific object:

# Re-verify replicated backups against their recorded checksums
php artisan backup:verify-integrity

# Restore a single information object from backup
php artisan backup:restore-io

Every restore is logged as a PREMIS event, preserving the audit trail.

Scheduled verification: Day-to-day, fixity runs from the in-app Scheduler (see Workflow Scheduler) - no per-task crontab editing is required once Heratio's task scheduler is active.

3. Virus Scanning

Heratio integrates with ClamAV to scan digital objects for malware.

Prerequisites

Install ClamAV on your server:

sudo apt install clamav clamav-daemon
sudo freshclam  # Update virus definitions

How Virus Scanning Works

Running Virus Scans

Via Web UI:

1. Navigate to Preservation -> Virus Scanning
2. View scan statistics and recent scan results
3. Run scans using the CLI commands shown

Via CLI:

# Scan up to 200 objects (default)
php artisan ahg:preservation-virus-scan

# Only scan objects with no prior scan record
php artisan ahg:preservation-virus-scan --unscanned --limit=100

# Show recent virus-scan history and exit
php artisan ahg:preservation-virus-scan --report

# Point at a specific ClamAV binary
php artisan ahg:preservation-virus-scan --clamav-binary=/usr/bin/clamscan

Quarantine

Infected files are moved to a quarantine area within the configured Heratio storage location, renamed with a timestamp and a .quarantine extension. Review quarantined files before permanent deletion.

4. Format Conversion

Convert files to archival-quality preservation formats using industry-standard tools.

Supported Conversions

How Format Conversion Works

Running Format Conversions

Via Web UI:

1. Navigate to Preservation -> Format Conversion
2. View conversion tools status and statistics
3. See recent conversions and any failures

Via CLI:

Conversion is driven by the Format Policy Registry (FPR). Each rule maps a source format to a target preservation or access format and the tool that performs it. Run normalization with ahg:normalize-existing:

# Normalize existing objects to their preservation master (queued)
php artisan ahg:normalize-existing

# Produce access (reference) derivatives instead
php artisan ahg:normalize-existing --purpose=access

# Only objects of a given MIME type, run inline rather than queued
php artisan ahg:normalize-existing --mime=image/jpeg --sync

# Cap the number of objects processed
php artisan ahg:normalize-existing --limit=50

For planned, approval-gated format migrations across a collection, use the migration runner:

# Run all active migration plans (or a specific plan)
php artisan ahg:preservation-migrate --plan-id=1 --limit=50

# Simulate without converting
php artisan ahg:preservation-migrate --dry-run

Output Location

Derivatives are attached as child digital objects of the original, each with its own SHA-256 checksum, and stored within the configured Heratio storage location. Originals are always preserved.

5. Backup Verification

Verify the integrity of backup files and archives.

How Backup Verification Works

Running Backup Verification

Via Web UI:

1. Navigate to Preservation -> Backup Verification
2. View verification statistics
3. See recent verification results

Via CLI:

# Re-verify replicated backups against their recorded SHA-256 checksums
php artisan backup:verify-integrity

Backup integrity is tracked in the replication ledger - each replica records its SHA-256, target driver, and verification status. See the Backup & Replication settings for target configuration.

6. Backup Replication

Replicate files to multiple backup targets for disaster recovery.

Supported Target Types

Managing Replication Targets

Via Web UI:

1. Go to Admin -> AHG Settings -> Preservation & Backup
2. Add, edit, or delete replication targets
3. Enable/disable targets as needed
4. View replication logs

Running Replication

Via CLI:

# Replicate unreplicated packages to all enabled targets (default limit 20/target)
php artisan ahg:preservation-replicate

# Replicate to a specific named target
php artisan ahg:preservation-replicate --target=offsite-s3 --limit=100

# Replicate a specific package
php artisan ahg:preservation-replicate --package-id=42

# Simulate without transferring
php artisan ahg:preservation-replicate --dry-run

Replication Workflow

7. PREMIS Events

Every preservation action is logged as a PREMIS event, creating a complete audit trail.

Event Types

Viewing Events

1. Go to Preservation -> PREMIS Events
2. Filter by:
   • Event type
   • Date range
   • Outcome (success/failure)
   • Object

8. Format Identification (PRONOM)

Heratio integrates with Siegfried to provide PRONOM-based format identification, the same identification method used by The National Archives (UK) and DROID.

What is PRONOM?

PRONOM is a technical registry maintained by The National Archives that contains information about file formats. Each format has a unique identifier called a PUID (PRONOM Unique Identifier), such as:

• fmt/43 - JPEG File Interchange Format 1.02
• fmt/353 - Tagged Image File Format (TIFF)
• fmt/276 - PDF 1.7
• fmt/96 - Hypertext Markup Language 4.0

Prerequisites

Install Siegfried on your server:

curl -sL "https://github.com/richardlehane/siegfried/releases/download/v1.11.1/siegfried_1.11.1-1_amd64.deb" -o /tmp/sf.deb
sudo dpkg -i /tmp/sf.deb

How Format Identification Works

Confidence Levels

Running Format Identification

Via Web UI:

1. Navigate to Preservation -> Format Identification
2. View Siegfried status and statistics
3. See identification coverage and confidence distribution
4. Review top identified formats and recent identifications

Via CLI:

# Identify up to 1000 unidentified objects (default)
php artisan ahg:preservation-identify

# Identify a single digital object
php artisan ahg:preservation-identify --digital-object-id=123

# Print the PRONOM risk distribution and exit
php artisan ahg:preservation-identify --risk

Benefits of PRONOM Identification

• Standardized - Uses international format registry
• Accurate - Signature-based detection, not just extensions
• Risk Assessment - Automatically assigns preservation risk levels
• Interoperability - Compatible with DROID, Archivematica, Preservica

Format Registry Population

The PRONOM format registry is populated automatically as objects are identified: each new PUID encountered during ahg:preservation-identify is recorded in the format registry with its name, version, MIME type, risk level, and preservation action. Run identification across your collection to keep the registry current; review the results under Preservation -> Format Registry.

Format data captured per PUID:

• Official format name and version
• MIME type
• Risk level (low / medium / high / unknown)
• Whether it is a preservation format, and the recommended action (retain or monitor)

9. Workflow Scheduler

The Workflow Scheduler allows you to configure and monitor automated preservation tasks through a visual interface, without requiring direct server access.

Accessing the Scheduler

1. Navigate to Preservation -> Scheduler
2. Or go directly to: /admin/preservation/scheduler

Default Schedules

Creating/Editing Schedules

1. Click New Schedule or Edit on an existing schedule
2. Configure:
   • Name - Descriptive name for the schedule
   • Workflow Type - What task to run
   • Cron Expression - When to run (e.g., 0 2 * * * for daily at 2 AM)
   • Batch Limit - Maximum objects to process per run
   • Timeout - Maximum runtime before abort
   • Notifications - Email alerts on failure

Running Workflows Manually

You can manually trigger any workflow from the UI:

1. Go to Preservation -> Scheduler
2. Click the Play button next to any schedule
3. View results in the Recent Runs table

CLI Scheduler Command

Due preservation workflows are dispatched by ahg:preservation-scheduler, which Heratio's task scheduler invokes for you. You can also run it manually:

# Run all due schedules
php artisan ahg:preservation-scheduler

# Force every schedule to run now, regardless of last run time
php artisan ahg:preservation-scheduler --force

# Simulate without executing
php artisan ahg:preservation-scheduler --dry-run

As long as Heratio's task scheduler is active (the standard single php artisan schedule:run cron entry), there is no need to add a crontab line per workflow.

10. Format Registry

The format registry tracks file formats and their preservation risk level. It is automatically populated during format identification with PRONOM PUIDs.

Risk Levels

Viewing Format Information

1. Go to Preservation -> Format Registry
2. View all known formats with PUIDs
3. Filter by risk level
4. See objects using each format
5. Click PUID to view details on PRONOM website

CLI Command Reference

All Preservation Commands

Common Options

Scheduling

The recommended approach is the in-app Workflow Scheduler (Preservation -> Scheduler), which runs all configured workflows automatically via Heratio's task scheduler. You only need the standard Laravel scheduler cron entry on the server:

# The single cron line that drives ALL Heratio scheduled tasks
* * * * * cd /usr/share/nginx/heratio && php artisan schedule:run >> /dev/null 2>&1

Individual commands from the table above can also be run manually on demand.

10. OAIS Packages (SIP/AIP/DIP)

The OAIS Packages feature allows you to create, manage, and export archival packages following the OAIS (Open Archival Information System) standard. Packages are created in BagIt format for maximum interoperability.

Package Types

Creating a Package via Web UI

1. Navigate to Digital Preservation -> OAIS Packages
2. Click Create Package
3. Fill in the package details:
   • Name - Descriptive name (e.g., "Annual Reports 2024 SIP")
   • Description - Brief explanation of contents
   • Package Type - SIP, AIP, or DIP
   • Originator - Creating organization
4. Click Create Package
5. Add digital objects to the package
6. Build the BagIt package
7. Validate the package checksums
8. Export to ZIP, TAR, or TAR.GZ

Package Workflow

CLI Commands

The full draft -> build -> validate -> export workflow is driven from the OAIS Packages UI. The CLI builds a package directly for a given information object:

# Build an AIP for an information object
php artisan ahg:preservation-package --type=aip --object-id=123

# Build a SIP
php artisan ahg:preservation-package --type=sip --object-id=123

# Build a DIP from an existing AIP
php artisan ahg:preservation-package --type=dip --from-aip=45 --object-id=123

# Include derivative files, and record who built it
php artisan ahg:preservation-package --type=aip --object-id=123 --include-derivatives --created-by="archivist@example.org"

BagIt Structure

Each package follows the BagIt specification:

<uuid>/
  bagit.txt           # BagIt declaration
  bag-info.txt        # Package metadata
  manifest-sha256.txt # Payload checksums
  tagmanifest-sha256.txt # Tag file checksums
  data/
    file1.pdf         # Payload files
    file2.tiff
    ...

Package Conversions

Note: SIP must be validated before converting to AIP. AIP must be validated before creating DIP.

Best Practices

Recommended Settings

1. Algorithm: Use SHA-256 for all new checksums
2. Coverage: Aim for 100% checksum coverage
3. Verification Frequency: Verify all files at least monthly
4. Virus Scanning: Scan all new uploads immediately
5. Format Conversion: Convert at-risk formats proactively
6. Replication: Maintain at least 2 backup copies in different locations

Preservation Strategy

Format Migration

Plan and execute format conversions to ensure long-term accessibility.

Understanding Format Risk

Obsolescence Report

View formats at risk in your repository:

Migration Pathways

Pre-defined conversion routes:

Creating a Migration Plan

Step 1: Review Obsolescence Report
        ↓
Step 2: Select formats to migrate
        ↓
Step 3: Choose target formats
        ↓
Step 4: Create migration plan
        ↓
Step 5: Get approval (if required)
        ↓
Step 6: Execute migration
        ↓
Step 7: Verify results

Migration Plan Status

Migration Best Practices