Changelog

CSV and JSON writers in the core. The zero-dependency TableWriter serializes the record table — TableWriter.ToCsv(table) / ToJson(table). CSV is a flat, RFC 4180-quoted summary; JSON carries the full record including text, metadata, redaction counts, warnings, and content hash.

Timestamps default to UTC, with an opt-in for local time. Modified is written in UTC (trailing Z). Pass utc: false to emit machine-local time with an explicit offset (e.g. 2026-07-19T15:30:00+05:30) so it stays unambiguous.

New package: Scrubkit.Parquet. Writes the table to Apache Parquet via Parquet.Net (ParquetTableWriter) for data-lake / analytics ingestion. net8.0-only; the core stays zero-dependency. Parquet always stores Modified as a UTC instant.

Per-file diagnostics hook. ReadOptions.OnDiagnostic is a dependency-free Action<ScrubDiagnostic> the core fires per file — a read event on success, or a warning code (extract-failed, skipped-content, text-clipped, stat-failed, hash-failed) on a problem. AddScrubkit bridges it to ILogger, so the core takes no logging dependency.

Opt-in SHA-256 per file. ReadOptions.ComputeContentHash populates FileRecord.ContentHash with the file's lower-case hex SHA-256, bounded by MaxBytesPerFile (default lowered to 10 MB; oversized files are stat-skipped and never read).

Reads .epub e-books. The OPF Title / Author / Subject become metadata and the XHTML spine (tags stripped, entities decoded) becomes text, in reading order. Zero dependencies beyond Scrubkit.Abstractions; .epub files come back as Document rows.

First-class DI integration. services.AddScrubkit(…) registers a configured FolderScrubber as a singleton for ASP.NET Core and worker hosts. Configure recursion, limits, add-on extractors, and an optional redactor via a ReadOptions hook (with an IServiceProvider overload for DI-resolved dependencies).

Published throughput number. A BenchmarkDotNet harness extracts on the order of ~12,000 files/sec (4-way parallel; ~4,900 sequential) over a mixed text-file corpus.

Reads .eml (MIME) email. The From / To / Cc / Subject / Date headers become metadata and the message body becomes text.

Understands real email. Multipart messages, base64 and quoted-printable encodings, common charsets, and RFC 2047 encoded-word headers — preferring text/plain, falling back to text/html. Attachments are skipped. Comes back as a TypeBucket = "Email" row.

Leads with extraction. The package Title, description, tags, and both READMEs were rewritten around offline text + metadata extraction for .NET.

Neutral extraction demo. The playground was refreshed into an extraction view with an updated screenshot (synthetic sample data only).

Clarified text-family formats. RTF / HTML / XML are read as raw text — markup is not stripped.

Offline-guarantee test. The build fails if either shipping assembly ever references a networking assembly.

netstandard2.0 runtime test project. Runs the netstandard2.0 build — its PolySharp polyfills and GetValueOrDefault shim — on the .NET 8 host, so the polyfilled paths are exercised at runtime, not just compiled.

Expanded coverage. Extraction, extension filtering, record fields, opt-in and custom redaction, and edge cases (empty folders, uncapped limits, corrupt Office files, a stat-failed guard). CI now publishes a test-status summary on every run.

Redesigned NuGet pages. A scannable highlights list, separate Install and Quick-start sections, a supported-formats table, and clearer dividers — plus a focused Scrubkit.Abstractions page.

Added a "Prepare a folder for RAG" recipe and a short guide to writing a custom IFileExtractor or IRedactor.

Enabled package validation. The build checks that the net8.0 and netstandard2.0 API surfaces stay compatible within each package.

Sharpened discoverability. Rewrote the description to lead with the problem it solves and refreshed the search tags.

Point it at a folder, get a clean table back. FolderScrubber walks a directory and returns one row per file — its text, metadata, and warnings — fully offline. Unreadable files never crash the run.

Reads the common formats. PDF, Word / Excel / PowerPoint (docx / xlsx / pptx), plain-text files (txt, md, csv, log, json, xml, html), and the camera/software info in image EXIF.

Handles big folders. Stream results as they're produced (ReadStreamAsync) and process several files at once (MaxDegreeOfParallelism), with output order preserved.

Extend it without forking. Plug in your own formats (IFileExtractor) or redaction (IRedactor); add-ons only need the tiny Scrubkit.Abstractions package.