Heratio Help Center article. Category: AI & Automation.

AI Services & NER - User Guide

Version 1.6.x | January 2026

Table of Contents

1. Introduction
2. Accessing AI Services Settings
3. Configuration Options
4. Named Entity Recognition (NER)
5. AI Summarization
6. Spell Checking
7. NER Review Dashboard
8. Batch Processing
9. Troubleshooting

Introduction

The AI Services module provides intelligent automation for archival description:

• Named Entity Recognition (NER): Automatically extract people, organizations, places, and dates
• AI Summarization: Generate summaries from PDF documents
• Spell Checking: Identify spelling errors in metadata

Workflow Overview

Accessing AI Services Settings

1. Log in as administrator
2. Navigate to Admin → AHG Settings → AI Services
3. Or go directly to: /admin/ahg-settings/ai-services

Configuration Options

API Configuration

NER Settings

Summarization Settings

Spell Check Settings

Named Entity Recognition (NER)

What is NER?

NER automatically identifies and classifies named entities in text into predefined categories.

Entity Types

How It Works

Text Sources

NER extracts text from multiple sources:

1. Metadata Fields

   • Title
   • Scope and Content
   • Archival History

2. Attached PDFs (when "Extract from PDFs" is enabled)

   • Uses pdftotext for text extraction
   • Limited to first 50,000 characters per document
   • PDFs must contain searchable text (not scanned images)

Viewing Extracted Entities

1. Navigate to a record's view page
2. Look for the Entities section in the sidebar
3. Entities are grouped by type (People, Organizations, Places, Dates)

AI Summarization

Overview

AI Summarization automatically generates concise summaries from PDF documents and saves them to the specified metadata field (typically Scope and Content).

Workflow

Requirements

• PDF must contain searchable text (not scanned images)
• Minimum 200 characters of extractable text
• For scanned documents, run OCR first

Best Practices

1. Review generated summaries - AI summaries should be reviewed for accuracy
2. OCR scanned documents - Run OCR before summarization for best results
3. Check historical documents - Names and places may need verification

Spell Checking

Overview

Spell checking identifies potential spelling errors in metadata fields using language-specific dictionaries.

Supported Languages

Fields Checked

By default, spell checking runs on:

• Title
• Scope and Content

Additional fields can be configured in settings.

Result Status

NER Review Dashboard

Accessing the Dashboard

Navigate to: /ner/review or Admin → NER Review

Dashboard Interface

Review Actions

Bulk Actions

• Approve Selected: Approve multiple entities at once
• Reject Selected: Reject multiple entities
• Export: Export entities to CSV for external processing

Linking to Authority Records

When approving a PERSON or ORG entity, you can link it to an existing authority record:

1. Click Approve on the entity
2. Search for existing authority record
3. Select match or create new
4. Entity is linked for future reference

Batch Processing

Overview

For large archives, batch processing via CLI is more efficient than processing records individually.

CLI Commands

NER Extraction

# Extract from all records that have no NER entities yet
php artisan ahg:ai-ner --unprocessed --limit=1000

# Set the reporting batch size while processing
php artisan ahg:ai-ner --unprocessed --limit=500 --batch=50

# Preview only (dry run)
php artisan ahg:ai-ner --unprocessed --dry-run --limit=10

# Read descriptions from a specific culture
php artisan ahg:ai-ner --unprocessed --culture=af --limit=100

Summarization

# Summarize records with empty scope_and_content
php artisan ahg:ai-summarize --all-empty --limit=100

# Summarize specific record
php artisan ahg:ai-summarize --object=12345

# Limit to one repository
php artisan ahg:ai-summarize --all-empty --repository=5 --limit=100

# Specify different target field
php artisan ahg:ai-summarize --all-empty --field=abstract --limit=100

Spell Check

# Check all records
php artisan ahg:ai-spellcheck --all --limit=100

# Check specific repository
php artisan ahg:ai-spellcheck --repository=5 --limit=500

# Use different culture
php artisan ahg:ai-spellcheck --all --culture=af --limit=100

Running Long Batches

For large-scale processing, use screen to run in the background:

# Start a screen session
screen -S batch_ner

# Run the batch
php artisan ahg:ai-ner --unprocessed --limit=100000

# Detach from screen: Ctrl+A, then D
# Reattach later: screen -r batch_ner

Monitoring Progress

# Quick status check
mysql -u root database -e "
SELECT COUNT(*) as processed FROM ahg_ner_extraction;
SELECT COUNT(*) as entities FROM ahg_ner_entity;
"

# Detailed progress
mysql -u root database -e "
SELECT 
    (SELECT COUNT(*) FROM ahg_ner_extraction) as processed,
    (SELECT COUNT(*) FROM ahg_ner_entity) as entities,
    (SELECT COUNT(*) FROM information_object WHERE id != 1) - 
    (SELECT COUNT(*) FROM ahg_ner_extraction) as pending;
"

Troubleshooting

Common Issues

No Entities Found

Symptoms: Records processed but no entities extracted

Possible Causes:

1. Empty metadata fields
2. No PDF attached
3. PDF is image-only (not searchable)
4. "Extract from PDFs" disabled

Solutions:

1. Enable "Extract from PDFs" in settings
2. Ensure PDFs contain searchable text
3. Run OCR on scanned documents first
4. Check record has content in metadata fields

"Text too short" Error

Symptoms: Summarization fails with "Text too short"

Cause: Document has less than 200 characters of text

Solution: This is normal for brief records - summarization is skipped. No action needed.

API Connection Error

Symptoms: "API error: HTTP 0" or timeout errors

Solutions:

1. Verify API URL in settings is correct
2. Check AI service is running: curl http://API_URL/health
3. Verify API key is correct
4. Increase timeout for large documents

Elasticsearch Errors

Symptoms: Errors when saving records after processing

Solutions:

1. Check ES running: systemctl status elasticsearch
2. Verify Elastica version matches Elasticsearch version
3. Rebuild index: php artisan ahg:search-populate

Summary Not Saving

Symptoms: "Processed" reported but no summary in record

Possible Cause: Field name mismatch

Solution: Ensure summary_field setting is scope_and_content (with underscores)

Quick Reference Card

URLs

CLI Commands

# NER
php artisan ahg:ai-ner --unprocessed --limit=N

# Summarize
php artisan ahg:ai-summarize --all-empty --limit=N
php artisan ahg:ai-summarize --object=ID

# Spell Check
php artisan ahg:ai-spellcheck --all --limit=N

Monitor Progress

SELECT COUNT(*) FROM ahg_ner_extraction;  -- Processed
SELECT COUNT(*) FROM ahg_ner_entity;       -- Entities

Entity Types

May 2026 update - 16 settings keys now wired

The form at /admin/ahgSettings/aiServices previously had 16 of 20 fields that saved to ahg_ner_settings but were never read by any consumer. v1.53.23 wired them through a new AhgAiServices\Support\AiServicesSettings helper. The four master gates - summarizer_enabled, spellcheck_enabled, translation_enabled, ner_enabled - now act as global kill switches that sit ABOVE the per-session ingest toggles. Both must be on for the step to run.

Processing mode

ai_services_processing_mode (local | cloud | hybrid). Cloud mode posts to a single hosted endpoint configured via ai_services_api_url + ai_services_api_key + ai_services_api_timeout, bypassing the per-provider config table at /admin/ai/llm-config. Use it when you want one stable hosted endpoint to handle every AI call instead of registering each provider individually.

Translation mode

translation_mode (mt | llm). The mt mode posts to mt_endpoint with mt_timeout (e.g. for an NLLB-200 sidecar) and falls through to the LLM round-trip on failure or when the endpoint is missing. llm mode skips the MT endpoint entirely and uses the configured LLM with a translation prompt.

Discovery / Qdrant fallback

The vector search service now falls back to qdrant_url, qdrant_collection, qdrant_model, qdrant_min_score (the keys exposed on the AI Services form) when the canonical semantic_qdrant_* settings are unset. So you don't have to know about both tables - the AI Services tile is enough. qdrant_min_score is sent to Qdrant as score_threshold and filters out hits below the floor.

Auto-extract on upload

New auto_extract_on_upload toggle - when on, file uploads of digital objects auto-trigger Donut document extraction. URL/FTP-linked objects bypass since they have no local file. Result lands in ahg_ai_inference with full PROV-O provenance (see "AI Inference Provenance - User Guide" for the trace endpoint and override workflow).

Master gate + per-session interplay

For ingest sessions (the wizard at /admin/ingest/...), every per-session process_* toggle is now AND-ed against its master gate:

Turning a master gate off silently disables the step across every ingest session, regardless of per-session settings. Useful for cost control and emergency disable.

Part of the Heratio platform | Updated May 2026