Scrubkit 2.0 Announcement
Major Architecture Release · v2.0.0

Announcing Scrubkit 2.0

A monumental architectural evolution. The core engine is now 100% zero-dependency, format extractors are completely modular, and test coverage scales to 575+ tests across .NET Standard 2.0, .NET 8 LTS, .NET 9, and .NET 10.

verified

Zero External Dependencies

The core Scrubkit package has shed all third-party dependencies. Built entirely on pure .NET BCL: Word, Excel, PowerPoint (OOXML), HTML, RTF, and Plain Text. Zero package bloat.

extension

Modular Add-On Packages

Heavy parsers are now decoupled into dedicated opt-in packages: Scrubkit.Pdf (PdfPig) and Scrubkit.Images (MetadataExtractor), giving you surgical control over application binary size.

checklist

575+ Automated Tests

Massive test expansion with comprehensive theory suites covering edge cases, ReDoS regex timeouts, span normalizations, and concurrent streams with 99%+ code coverage.

architecture

The Architecture Evolution

In Scrubkit 1.x, the core package bundled PdfPig (for PDF text) and MetadataExtractor (for EXIF image metadata). While convenient, it pulled third-party assemblies into applications that only needed plain text, Office, or email processing. In Scrubkit 2.0, the core is 100% clean BCL, with every heavy parser extracted into its own package.

Scrubkit 1.x (Monolithic Core) 2 Third-Party Deps
  • ✔ Office OOXML (docx, xlsx, pptx)
  • ✔ HTML & RTF clean-text
  • ✔ Plain text (txt, md, json, etc.)
  • ⚠ PdfPig (transitive in core)
  • ⚠ MetadataExtractor (transitive in core)
Scrubkit 2.0 (Modular Core) 0 External Deps
  • ✔ Scrubkit (Core): Zero third-party dependencies
  • + Scrubkit.Pdf: Dedicated add-on via PdfPig
  • + Scrubkit.Images: Dedicated add-on via MetadataExtractor
  • + Scrubkit.All: Meta-package with .AddAllExtractors()
code_blocks

Fluent Extractor Registry

Register only the formats your application needs, or use Scrubkit.All to register the full suite in a single line.

Option A: Selective Registration (Pick Exactly What You Need) C#
using Scrubkit;

// Register only the extractors needed for your pipeline
var options = new ReadOptions()
    .AddPdf()           // from Scrubkit.Pdf
    .AddImages()        // from Scrubkit.Images
    .AddEmail()         // from Scrubkit.Email
    .AddOpenDocument(); // from Scrubkit.OpenDocument

var scrubber = new FolderScrubber(options);
await foreach (var record in scrubber.ReadStreamAsync(@"C:\Data"))
{
    Console.WriteLine($"{record.Name} [{record.TypeBucket}] — {record.Text.Length} chars");
}
Option B: Scrubkit.All (One-Line Complete Registration) C#
using Scrubkit;

// Scrubkit.All provides the convenience method .AddAllExtractors()
var options = new ReadOptions().AddAllExtractors();

var scrubber = new FolderScrubber(options);
IReadOnlyList<FileRecord> table = await scrubber.ReadAsync(@"C:\KnowledgeBase");
sync_alt

Upgrading from 1.x to 2.0 (Migration Guide)

Upgrading takes less than a minute. Choose the path that matches your current setup:

If you use Scrubkit.All:

Simply update your package reference and call .AddAllExtractors():

// Old 1.x:
var scrubber = new FolderScrubber();

// New 2.0:
var scrubber = new FolderScrubber(new ReadOptions().AddAllExtractors());

If you only need Office and Plain Text:

You don't need any add-ons! Enjoy a smaller deployment with zero external dependencies:

// Zero external dependencies out of the box!
var scrubber = new FolderScrubber();
var records = await scrubber.ReadAsync(@"C:\OfficeDocs");

If you need PDF or Image extraction with core Scrubkit:

Install the dedicated add-on package and register it:

// In your project:
// dotnet add package Scrubkit.Pdf

var options = new ReadOptions().AddPdf();
var scrubber = new FolderScrubber(options);
view_module

The Complete 14-Package Family

Core

Scrubkit

Zero-dependency engine (Office, HTML, RTF, Text).

Contracts

Scrubkit.Abstractions

Lightweight interfaces and data records.

Add-on

Scrubkit.Pdf

PDF document text & metadata via PdfPig.

Add-on

Scrubkit.Images

Image EXIF metadata via MetadataExtractor.

Add-on

Scrubkit.Email

MIME .eml and Outlook .msg parsing.

Add-on

Scrubkit.OpenDocument

LibreOffice .odt, .ods, .odp.

Add-on

Scrubkit.Epub

EPUB e-book clean-text parser.

Add-on

Scrubkit.LegacyOffice

Pre-2007 binary .doc, .xls, .ppt.

Integration

Scrubkit.Extensions.DependencyInjection

services.AddScrubkit(...) DI helper.

AI

Scrubkit.Extensions.MicrosoftExtensionsAI

Local PII & secret redaction middleware.

AI

Scrubkit.Extensions.SemanticKernel

Vector store & memory ingestion extensions.

Output

Scrubkit.Parquet

Columnar Parquet data lake table export.

CLI

Scrubkit.Tool

Zero-code global command-line tool.

Bundle

Scrubkit.All

All-in-one meta-package with .AddAllExtractors().