Announcing Scrubkit 2.0
A monumental architectural evolution. The core engine is now 100% zero-dependency, format extractors are completely modular, and test coverage scales to 575+ tests across .NET Standard 2.0, .NET 8 LTS, .NET 9, and .NET 10.
Zero External Dependencies
The core Scrubkit package has shed all third-party dependencies. Built entirely on pure .NET BCL: Word, Excel, PowerPoint (OOXML), HTML, RTF, and Plain Text. Zero package bloat.
Modular Add-On Packages
Heavy parsers are now decoupled into dedicated opt-in packages: Scrubkit.Pdf (PdfPig) and Scrubkit.Images (MetadataExtractor), giving you surgical control over application binary size.
575+ Automated Tests
Massive test expansion with comprehensive theory suites covering edge cases, ReDoS regex timeouts, span normalizations, and concurrent streams with 99%+ code coverage.
The Architecture Evolution
In Scrubkit 1.x, the core package bundled PdfPig (for PDF text) and MetadataExtractor (for EXIF image metadata). While convenient, it pulled third-party assemblies into applications that only needed plain text, Office, or email processing. In Scrubkit 2.0, the core is 100% clean BCL, with every heavy parser extracted into its own package.
- ✔ Office OOXML (docx, xlsx, pptx)
- ✔ HTML & RTF clean-text
- ✔ Plain text (txt, md, json, etc.)
- ⚠ PdfPig (transitive in core)
- ⚠ MetadataExtractor (transitive in core)
- ✔ Scrubkit (Core): Zero third-party dependencies
- + Scrubkit.Pdf: Dedicated add-on via PdfPig
- + Scrubkit.Images: Dedicated add-on via MetadataExtractor
- + Scrubkit.All: Meta-package with
.AddAllExtractors()
Fluent Extractor Registry
Register only the formats your application needs, or use Scrubkit.All to register the full suite in a single line.
using Scrubkit;
// Register only the extractors needed for your pipeline
var options = new ReadOptions()
.AddPdf() // from Scrubkit.Pdf
.AddImages() // from Scrubkit.Images
.AddEmail() // from Scrubkit.Email
.AddOpenDocument(); // from Scrubkit.OpenDocument
var scrubber = new FolderScrubber(options);
await foreach (var record in scrubber.ReadStreamAsync(@"C:\Data"))
{
Console.WriteLine($"{record.Name} [{record.TypeBucket}] — {record.Text.Length} chars");
}
using Scrubkit;
// Scrubkit.All provides the convenience method .AddAllExtractors()
var options = new ReadOptions().AddAllExtractors();
var scrubber = new FolderScrubber(options);
IReadOnlyList<FileRecord> table = await scrubber.ReadAsync(@"C:\KnowledgeBase");
Upgrading from 1.x to 2.0 (Migration Guide)
Upgrading takes less than a minute. Choose the path that matches your current setup:
If you use Scrubkit.All:
Simply update your package reference and call .AddAllExtractors():
// Old 1.x:
var scrubber = new FolderScrubber();
// New 2.0:
var scrubber = new FolderScrubber(new ReadOptions().AddAllExtractors());
If you only need Office and Plain Text:
You don't need any add-ons! Enjoy a smaller deployment with zero external dependencies:
// Zero external dependencies out of the box!
var scrubber = new FolderScrubber();
var records = await scrubber.ReadAsync(@"C:\OfficeDocs");
If you need PDF or Image extraction with core Scrubkit:
Install the dedicated add-on package and register it:
// In your project:
// dotnet add package Scrubkit.Pdf
var options = new ReadOptions().AddPdf();
var scrubber = new FolderScrubber(options);
The Complete 14-Package Family
Scrubkit
Zero-dependency engine (Office, HTML, RTF, Text).
Scrubkit.Abstractions
Lightweight interfaces and data records.
Scrubkit.Pdf
PDF document text & metadata via PdfPig.
Scrubkit.Images
Image EXIF metadata via MetadataExtractor.
Scrubkit.Email
MIME .eml and Outlook .msg parsing.
Scrubkit.OpenDocument
LibreOffice .odt, .ods, .odp.
Scrubkit.Epub
EPUB e-book clean-text parser.
Scrubkit.LegacyOffice
Pre-2007 binary .doc, .xls, .ppt.
Scrubkit.Extensions.DependencyInjection
services.AddScrubkit(...) DI helper.
Scrubkit.Extensions.MicrosoftExtensionsAI
Local PII & secret redaction middleware.
Scrubkit.Extensions.SemanticKernel
Vector store & memory ingestion extensions.
Scrubkit.Parquet
Columnar Parquet data lake table export.
Scrubkit.Tool
Zero-code global command-line tool.
Scrubkit.All
All-in-one meta-package with .AddAllExtractors().