GraphQL-core: Denial of service via comment tokens bypassing the parser's max_tokens limit

### Summary The parser's `max_tokens` limit — the documented defence against oversized GraphQL documents — did not count comment tokens. A document padded with comments could therefore pass a `max_tokens` check of any size while still allocating one `Token` object per comment, all of which stay reachable for the lifetime of the parsed document. A 38 MB comment-padded payload allocated roughly 92 MB of additional resident memory with `max_tokens=5` in force. ### Details `Lexer.lookahead()` reads tokens one at a time and skips over comments, returning only the first non-comment token. Each skipped comment is nevertheless fully tokenized, allocated as a `Token` object, and linked into the persistent token list via `token.next` / `token.prev` — the chain that powers source-location reporting, and which is retained as long as the document is. `Parser.advance_lexer()` incremented the token counter only for the tokens the lexer actually returned. Comments were therefore invisible to the accounting, and `max_tokens` bounded only the *non-comment* tokens. The practical consequence is a gap between what users reasonably believe `max_tokens` protects and what it did protect. The limit was not broken for ordinary documents — real tokens were still capped correctly — so the shortfall was not observable without specifically testing comment padding. As with the companion issue, the payload compresses extremely well (`# c\n` repeated), so body-size limits and WAF inspection applied to the compressed request do not see its true size, while the framework decompresses it in memory before the parser is reached. ### Proof of concept ```python import gc import resource from graphql import parse # 100,000 comments, a 5-token query, and max_tokens=5 — accepted. query = "# c\n" * 100_000 + "{ a }" parse(query, max_tokens=5) # no error raised # Memory amplification. big = "# padding comment used to amplify memory per token\n" * 500_000 + "{ a }" gc.collect() before = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss parse(big, max_tokens=5) # ~38 MB source after = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss print(f"RSS delta: +{(after - before) / 1024:.0f} MB") ``` Measured on GraphQL-core 3.2.8 / CPython 3.12 / Linux x86_64: the 391 KB payload parsed in 127 ms, and the 38.1 MB payload produced an RSS delta of **+92 MB** — about 2.4× the source size, on top of the source string itself. ### Impact Denial of service through memory exhaustion. A modest number of concurrent requests can push a worker past a typical 256 MB or 512 MB container limit and trigger an OOM kill. No authentication is required, and no data is disclosed or modified. Depth limiters and complexity analyzers do not help: they run on the AST after parsing has already succeeded, and the document contains only five real tokens. ### Patches Fixed in **3.2.12** and **3.3.0rc1**. `Parser.advance_lexer` now counts every token consumed, including the comment tokens the lexer skips between the previously focused token and the new one, and including the advance onto EOF so that trailing comments are covered. `max_tokens` therefore now bounds every `Token` the parser allocates. Note the behaviour change: **comments now count toward `max_tokens`.** A comment-heavy document that was previously accepted may now be rejected. If you parse documents containing substantial comments — SDL files with documentation comments, for instance — and you set `max_tokens`, you may need to raise the limit. The `token_count` attribute of a parsed document likewise now includes comments. ### Workarounds If you cannot upgrade, apply a request body-size limit to the **decompressed** body before the document reaches `parse()`. Such a limit is worth keeping after upgrading as well: `max_tokens` is one guard among several and bounds the token count, not the overall cost of a request. ### References - GraphQL specification § 2.1.4 — Comments ### Credits Reported by @BZHunt, who also reviewed the fixes before release. In memory of ZaX, who was the driving force behind this discovery.