Tobias Krug
All writing

A PDF that looks right and is wrong

Under XeLaTeX, Inter's contextual alternates turn a minus sign into a Private Use Area code point. The page looks fine; copy, search and parsers get no minus. The fix, and how to test for it.

Venture7 min readFacts checked

Contents

Set −123,45 in Inter with XeLaTeX and the PDF shows exactly that. Pull the text back out and what arrives is not a minus sign followed by digits. Every extractor I tried hands back a Private Use Area character in place of the minus, and no number parser will accept it. Ask poppler for Latin-1 or ASCII output (pdftotext -enc Latin1) and the minus disappears entirely, leaving a clean, positive 123,45.

Nothing on the page gives it away. The PDF looks right and is wrong for anyone who copies, searches, indexes or parses it. For a document full of figures, that is most of the people who matter.

schriftsatz, a Markdown-to-PDF tool I publish under MIT through CronAI, is built around two defects: this one, and PDFs with no structure a machine can read. It is shipped, at v0.3.0.

A minus sign only one font understands

Inter has a calt (contextual alternates) feature. Next to capitals or figures it swaps the minus, hyphen and parentheses for .case variants: the same marks, raised to suit cap height. On the page, it is a nice touch.

Those variants do have code points, but in the Private Use Area. Here are the plain minus, the raised one and its tabular-width twin in the Inter 4.000 that ships with TeX Live 2026:

import subprocess
from fontTools.ttLib import TTFont

path = subprocess.check_output(["kpsewhich", "Inter-Regular.otf"], text=True).strip()
cmap = TTFont(path).getBestCmap()
by_name = {name: hex(cp) for cp, name in cmap.items()}
print(by_name["minus"], by_name["minus.case"], by_name["minus.case.tf"])
# 0x2212 0xe09e 0xee6b

In the XeLaTeX pipeline, xdvipdfmx writes the PDF, ToUnicode table included, and it fills that table from the font. For the raised minus it writes U+EE6B. Strictly, the glyph drawn is minus.case and U+EE6B is its tabular twin's code point; either way, the answer is in the Private Use Area. That the glyph began as a minus sign was known when the line was shaped, and it never reaches ToUnicode. Nothing is unencoded: the mapping points at a code point the font really defines, one that means nothing outside it.

The repository documents the defect in a write-up of the Inter calt defect and ships a reproducer next to it. The write-up argues that Inter is not the odd one out: under XeLaTeX, any font that gives its alternates Private Use Area code points does the same. I have only tested Inter.

Here is the reproducer, extracted on 2026-10-14 with poppler 26.10.0, pypdf 6.20.1 and Apple's PDFKit (macOS 26.7.1). All three return the same string; whitespace is collapsed and the Private Use Area characters are escaped:

source:    Materialaufwand −123,45 · (§ 4 Abs. 2 MusterG, ¼) −6.789,01 · Test (Klammer) −42
extracted: Materialaufwand \uee6b123,45 · (§ 4 Abs. 2 MusterG, ¼\uee4f \uee6b6.789,01 · Test \uee4eKlammer) \uee6b42

Look at which parentheses survive: the one before § and the one after Klammer, the two that are not next to a capital or a figure. That asymmetry is the tell. And float("\uee6b123.45") raises ValueError, which is the friendlier failure. The quieter one is the Latin-1 output from the top: a PDF without the defect comes out there as -123,45, this one as a positive 123,45.

Two fixes that look right

RawFeature={-case} is the intuitive guess, because the substituted glyphs are named .case. But calt drives the substitution, not case, and the output does not change.

\XeTeXgenerateactualtext=1 is the convincing one. It wraps text runs in /ActualText spans, poppler honours them, and pdftotext returns −123,45 with calt still on. Typography kept, test green.

Except that /ActualText is an optional hint layered on top of the unchanged mapping. pypdf does not honour it, nor does PDFKit, and both still get U+EE6B. That fix repairs the extractor you happened to test with and leaves the text layer as wrong as it was.

The fix, and where it has to go

\setmainfont{Inter-Regular.otf}[Numbers={Proportional,Lining}, RawFeature={-calt}]

With calt off the substitution never happens. The glyph is U+2212, the cmap and ToUnicode agree, and no reader has to implement anything optional. The cost is Inter's case-sensitive punctuation: beside capitals, minus signs and parentheses stay at their lowercase height instead of being raised to match. The write-up compared 300 dpi A4 renders of both: 2,106 of 26 MB of raster bytes differ, about 0.008%. On XeLaTeX, for a document with numbers in it, I take that trade every time.

Getting the option to the font was a second bug. schriftsatz sets \defaultfontfeatures{RawFeature={-calt}} in a header fragment, which applies to every font loaded after it. But pandoc's template emits \setmainfont well before the -H header includes (checked with pandoc 3.12.1). So a font named in front matter, the documented way to choose a typeface, was loaded before the fix arrived, and the defect came back in full.

The test suite missed it because it set its font in a second header file, the one ordering where the default does apply.

A small Lua filter now also appends RawFeature={-calt} to the document's own font options (mainfontoptions and its sans and mono siblings), and leaves an explicit calt setting alone.

If changing engines is an option: LuaLaTeX does not have the problem, and not because it skips the feature. Built with LuaLaTeX, the reproducer uses the same raised glyphs, and the PDF's ToUnicode maps them to U+2212 and plain parentheses. It passes schriftsatz verify.

Test the text, not the picture

tests/calt-mwe/run.sh builds all four variants and asserts what each extractor returns, the failing cells included. An excerpt from my run, three poppler lines and one pypdf line:

poppler (pdftotext):
  ok   unfixed drops the minus (control)              absent −123,45
  ok   -case does NOT fix it (control)                absent −123,45
  ok   -calt fixes it                                 present −123,45

pypdf:
  ok   actualtext FAILS under pypdf (the whole point) absent −123,45

The controls are not decoration. If a build quietly falls back to a font without the feature, every "fixed" assertion still passes, and only the negative controls notice that the defect stopped reproducing. CI has a step called "Inter must resolve, or the calt suite tests nothing" for the same reason. The assertions strip whitespace first, because pypdf inserts spaces derived from glyph positions, and then look for the exact string −123,45, U+2212 included.

A check for the same defect ships as schriftsatz verify, for PDFs it did not build: a scan of poppler's output for Private Use Area code points, and a character-level comparison with pypdf. Trimmed of its note that the file is untagged:

$ schriftsatz verify unfixed.pdf
FAIL  unfixed.pdf: text layer contains Private Use Area codepoints (U+EE4E U+EE4F U+EE6B)
      the font mapped glyphs to codepoints that carry no meaning
      outside it — see docs/inter-calt-tounicode.md
ok    two extractors agree

The caveat: my own docs are out of date

The write-up says the two extractors fail in opposite directions. poppler discards Private Use Area code points, so the minus silently disappears and −123,45 becomes 123,45; pypdf passes them through. That disagreement was supposed to be the signature of the defect.

On 2026-08-23 I opened schriftsatz issue #63: on poppler 26.08.0, both extractors emit U+EE6B. My run on 26.10.0 shows the same, which is why verify above reports "two extractors agree" about a broken file.

The write-up is not the only stale place. The README, the code comments and the label on the first test line above say the same thing. That label, "unfixed drops the minus", describes a deletion my run does not show; the assertion under it only checks that U+2212 is absent, which is why it still passes.

What that does and does not break:

  • The fix and the suite are unaffected. The assertions check whether a real U+2212 is present, not how it went missing, so they hold either way.
  • verify still exits 1 on the unfixed file, because the Private Use Area scan over poppler's output catches it.
  • #63 calls the cross-check the weaker test. My run says it is not redundant. On the \XeTeXgenerateactualtext file poppler honours /ActualText, the scan finds nothing, and the cross-check is the only thing that fails. Trimmed to the minus signs:
$ schriftsatz verify actualtext.pdf
FAIL  actualtext.pdf: extractors disagree on characters (poppler vs pypdf)
      a text layer that depends on the reader is not a text layer
      U+2212 '−' ×3, seen only by poppler
      U+EE6B '\uee6b' ×3, seen only by pypdf

That cross-check needs uv, or a python3 with pypdf installed, and the brew cask installs neither. Without them, verify notes that only one extractor is available and passes this file.

#63 assumed poppler had changed, and I can't find that it did. With default settings, five poppler releases from 22.02 to 26.10 all return U+EE6B in place of the minus. The deletion appears only when pdftotext is asked for a non-Unicode output encoding, and the output the write-up records matches what pdftotext -enc Latin1 prints, character for character apart from whitespace, not what the plain pdftotext mwe.pdf - shown with it prints. The write-up's own status line names 26.08.0, the same version #63 tested. The likelier story is that the original measurement went through something that could not carry the character, not that poppler changed. #63 is where I'll pin that down.

What carries over

A document a machine will read needs a test on its text layer, not a screenshot diff. In practice:

  • Extract with two independent tools and assert on exact code points.
  • Keep negative controls, so a fixture that stops reproducing the defect fails loudly instead of passing quietly.
  • Assert the outcome you care about, not the mechanism of failure. The assertions here held up when the explanation behind them turned out to be wrong; their labels and the prose around them did not.
  • Record the version, the exact command and the raw output next to every measurement. The version in the write-up is what exposed the contradiction with #63; the output it records is what pointed at Latin-1.
  • When a measurement stops reproducing, try older versions before concluding the tool changed. #63 skipped that step.

schriftsatz v0.3.0 installs with brew install cronai-labs/tap/schriftsatz; install uv as well, or verify runs without its cross-check. One more caveat: the macOS binaries are not signed with a Developer ID or notarised, and the cask strips the quarantine attribute after install, which bypasses Gatekeeper rather than passing it. That is issue #19, signing and notarisation, still open.