💡 Full working example available on GitHub: parse-docx-to-markdown-chunks-dotnet

The Old Way Was Painful

Heading-based Markdown chunking is a .NET retrieval prep path that turns Office and PDF files into section-sized text before you embed. Without it, teams often dump an entire Word file into a single vector. The cover paragraph and the duty clause share one embedding, so cosine search ranks the wrong section. I hit that on a three-page internal guide: every query pulled the title block first because the index never saw the # / ## structure the authors already wrote.

Hand-rolling a DOCX-to-Markdown converter means walking Open XML styles, mapping list levels, and still failing when the next upload is a PDF whose formatted Markdown path returns null. Fixed-size token windows ignore those headings. Overnight corpora then mix empty shells with huge manuals because nobody gated on page count or size first.

The real cost of whole-file embeddings: wrong-section answers, wasted extract spend on junk uploads, and no chunk-01.md artifacts operators can open during a bad batch.

There’s a Better Way

GroupDocs.Parser for .NET replaces that scramble with four focused calls: GetDocumentInfo, GetFormattedText(FormattedTextMode.Markdown), GetText when Markdown is null, and a small heading walker that writes chunk-NN.md. The same Parser type covers Word DOCX parse and PDF text fallback, so you keep one C# service path.

You will need:

  • .NET 8 SDK
  • GroupDocs.Parser 26.10.0 (temporary license)
  • A Word DOCX (the sample is 3 pages / 14820 bytes) and optionally a PDF
dotnet add package GroupDocs.Parser --version 26.10.0

The New Way: Inspect, Extract, Fall Back, Split

Here is the same retrieval prep after the switch. The console sample under parse-docx-to-markdown-chunks-dotnet writes info, Markdown, PDF text, and four heading chunks for the shipped DOCX.

Step 1 - Gate with GetDocumentInfo

Ask for file type, page count, and size before you pay for a full extract.

using var parser = new Parser(sourcePath);
var info = parser.GetDocumentInfo();

var sb = new StringBuilder();
sb.AppendLine($"Source: {Path.GetFileName(sourcePath)}");
sb.AppendLine($"File type: {info?.FileType}");
sb.AppendLine($"Pages: {info?.PageCount}");
sb.AppendLine($"Size: {info?.Size} bytes");
File.WriteAllText(outputPath, sb.ToString());

Log those facts beside the job id. On the sample DOCX you should see a Microsoft Word Open XML Document, 3 pages, and 14820 bytes.

Step 2 - Prefer Markdown from GetFormattedText

For Word and formats that support formatted extraction, keep ATX headings for the splitter.

using var parser = new Parser(sourcePath);
using TextReader? reader = parser.GetFormattedText(
    new FormattedTextOptions(FormattedTextMode.Markdown));

if (reader == null)
{
    return null;
}

string markdown = reader.ReadToEnd();
File.WriteAllText(outputPath, markdown, Encoding.UTF8);
return markdown;

Null is a signal to switch paths, not a crash. Write UTF-8 so chunk tools and git diffs stay readable.

Step 3 - Fall back with GetText for PDF

When Markdown mode returns null, keep the file in the pipeline with a plain text layer.

using var parser = new Parser(sourcePath);
using TextReader? reader = parser.GetText();
if (reader == null)
{
    return null;
}

string text = reader.ReadToEnd();
File.WriteAllText(outputPath, text, Encoding.UTF8);
return text;

Expect .txt, not invented # headings. The sample host writes Result/sample-pdf.txt for the PDF branch.

Step 4 - Split Markdown on # and

Walk lines; flush when a new ATX # or ## appears; write chunk-NN.md.

var chunks = new List<string>();
var current = new StringBuilder();
using (var reader = new StringReader(markdown))
{
    string? line;
    while ((line = reader.ReadLine()) != null)
    {
        bool isHeading = line.StartsWith("# ", StringComparison.Ordinal)
            || line.StartsWith("## ", StringComparison.Ordinal);
        if (isHeading && current.Length > 0)
        {
            chunks.Add(current.ToString().TrimEnd());
            current.Clear();
        }
        current.AppendLine(line);
    }
}

Delete stale chunk-*.md files first. Deeper ### lines stay inside the current section. The sample DOCX yields four files under Result/chunks/.

When should I replace whole-file embeddings with this path?

Replace whole-file embeddings when intake mixes Word DOCX and PDF and the index needs section-sized Markdown. Run GetDocumentInfo first, try Markdown for Office files, call GetText when the formatted reader is null, and split only when # / ## markers exist. Leave unstructured PDF text alone until you add a separate chunk policy. That order matches the sample console app and keeps failure modes explicit for operators.

What Changed for Operators

Before After
One embedding per file One embedding per heading block
Custom Open XML Markdown GetFormattedText Markdown mode
PDF dropped on null formatted reader GetText writes a text artifact
No cheap pre-check Page count and size gate work

Dispose every Parser and TextReader. Apply a license or temporary license before overnight volume. Treat uploads as untrusted input and parse in an isolated worker.

Try It on Your Own Files

Clone parse-docx-to-markdown-chunks-dotnet, restore GroupDocs.Parser 26.10.0, and run against documents/sample.docx. Then point the same four methods at one of your manuals and open the generated chunk-*.md files.