feat: add opt-in AIA certificate chain fetching (tlsAIAFetch)

[vtorosyan]

Original PR and review comments are here: #5942

What?

Go's crypto/tls does not fetch missing intermediate certificates via the Authority Information Access (AIA) extension; browsers do. Servers that send only a leaf certificate and rely on AIA to fill in the chain cause k6 TLS handshakes to fail with "x509: certificate signed by unknown authority" even when the chain is technically valid.

This commit adds a new tlsAIAFetch option (env: K6_TLS_AIA_FETCH) that enables AIA chain fetching on a per-test basis. When enabled, k6 intercepts failed chain verification, follows IssuingCertificateURL fields (BFS, up to 5 hops), retries verification with any fetched intermediates, and then restores hostname verification — exactly what browsers do.

Implementation details:

• lib/netext/aia.go: core WrapTLSConfigForAIAFetching function; uses a separate http.Client (not measured as test traffic) with a 10s timeout; process-level sync.Map cache so repeated handshakes pay the HTTP cost at most once per AIA URL; circular-reference protection via seen map
• lib/options.go: new TLSAIAFetch null.Bool option (default off)
• internal/js/runner.go: opt-in wiring — wraps the per-VU tls.Config when tlsAIAFetch: true; no-op when insecureSkipTLSVerify is also true
• internal/js/modules/k6/grpc/client.go: fix buildTLSConfig to copy VerifyPeerCertificate and VerifyConnection callbacks from the parent config, so AIA verification is not silently bypassed for gRPC connections that supply explicit TLS options (cacerts/cert/key)

Tests:

• lib/netext/aia_test.go: 10 parallel unit tests covering happy path, complete chain passthrough, disabled/plain-Go regression, InsecureSkip- Verify passthrough, unreachable URL, fetch timeout, HTTP error, malformed cert, circular AIA references, and hostname-mismatch enforcement
• internal/js/runner_aia_test.go: full VU integration test (Source and Archive runners) verifying the enabled/disabled toggle end-to-end

Checklist

• I have performed a self-review of my code.
• I have commented on my code, particularly in hard-to-understand areas.
• I have added tests for my changes.
• I have run linter and tests locally (make check) and all pass.

Checklist: Documentation (only for k6 maintainers and if relevant)

Please do not merge this PR until the following items are filled out.

• I have added the correct milestone and labels to the PR.
• I have updated the release notes: link
• I have updated or added an issue to the k6-documentation: grafana/k6-docs#NUMBER if applicable
• I have updated or added an issue to the TypeScript definitions: grafana/k6-DefinitelyTyped#NUMBER if applicable

Related PR(s)/Issue(s)

Fixes #5941

[vtorosyan]

I ran again the end-to-end test chain in https://github.com/grafana/sm-k6-tls-test to ensure nothing changed against the previous PR, results are clean.

⏺ Full results against vt/tls_fix_v2:

  ┌────────────────────────────────────────────────────────┬─────────────────────────┐
  │                        Scenario                        │         Result          │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ aia-chain (server sends only leaf, requires AIA fetch) │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ complete-chain (full chain, AIA not needed)            │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ self-signed (should fail without skip)                 │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ self-signed/skip (with insecureSkipTLSVerify)          │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ insecure-skip                                          │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ wrong-hostname (should fail without skip)              │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ wrong-hostname/skip                                    │ PASS                    │
  ├────────────────────────────────────────────────────────┼─────────────────────────┤
  │ badssl-aia (real-world, incomplete-chain.badssl.com)   │ PASS (from earlier run) │

[szkiba]

@vtorosyan for me it seems ok, but to be sure I asked claude for review. Here is the output:

Blocker — gRPC ignores user-supplied CA when tlsAIAFetch is enabled

File: internal/js/modules/k6/grpc/client.go:172

buildTLSConfig now copies VerifyPeerCertificate / VerifyConnection from the
parent (VU) config:

tlsCfg := &tls.Config{
    ...
    RootCAs:               cp, // user-supplied cacerts
    VerifyPeerCertificate: parentConfig.VerifyPeerCertificate,
    VerifyConnection:      parentConfig.VerifyConnection,
}

When tlsAIAFetch: true, the parent config is the AIA-wrapped config, so its
VerifyPeerCertificate is the closure built in lib/netext/aia.go. That closure
reads its captured config's RootCAs, not the gRPC config's:

// lib/netext/aia.go:126
roots := cfg.RootCAs   // cfg = the wrapped VU config, NOT grpc's tlsCfg

So the cp pool that gRPC builds from the user's cacerts
(grpc/client.go:154-159) is never consulted during verification. Chain
verification runs against the wrapped VU config's RootCAs (typically nil → system
pool).

Failure scenario: A gRPC test against a server whose chain roots in a private
CA, supplied via cacerts, with options.tlsAIAFetch = true. Verification uses
the system pool instead of the provided CA and the handshake fails with
x509: certificate signed by unknown authority — even though the correct CA was
given.

Why it matters: silent, and it makes two supported features mutually
incompatible. A user enabling AIA globally silently breaks every gRPC call that
relies on a custom CA.

Suggested fix: the AIA callback must verify against the config it is attached
to, not a config captured at wrap time. Options:

• Re-wrap the gRPC tlsCfg with WrapTLSConfigForAIAFetching after setting
  RootCAs: cp (and don't copy the parent callbacks), so the closure captures
  the gRPC config. Cleanest.
• Or don't copy VerifyPeerCertificate/VerifyConnection in buildTLSConfig
  and re-apply AIA wrapping explicitly when the parent had it enabled.

Add a gRPC-path test covering cacerts + tlsAIAFetch: true.

---

Should fix — PKCS#7 (.p7c) AIA responses are silently rejected

File: lib/netext/aia.go:287

fetchCertFromAIAURL only handles raw DER and PEM:

if cert, parseErr := x509.ParseCertificate(body); parseErr == nil {
    return cert, nil
}
block, _ := pem.Decode(body)
if block == nil {
    return nil, errors.New("AIA response is neither valid DER nor PEM")
}

Many public CAs serve the issuing certificate as PKCS#7 / CMS
(application/pkcs7-mime, .p7c). Those responses parse as neither DER nor PEM,
so the intermediate is dropped and the chain stays incomplete — the exact case
this feature exists to fix.

Failure scenario: Server's IssuingCertificateURL points to a .p7c file.
Fetch succeeds (HTTP 200), parse fails, intermediate silently skipped, handshake
still fails with unknown authority. The user sees no clear reason why AIA
"didn't work".

Suggested fix: add a PKCS#7 fallback after the DER/PEM attempts (extract the
certificate(s) from the CMS bundle and add them). If PKCS#7 support is
intentionally out of scope, document the limitation prominently and log at Warn
when a response looks like PKCS#7 so users can diagnose it.

[vtorosyan]

@szkiba thank you for the feedback! I have pushed some changes:

• buildTLSConfig no longer copies the parent's VerifyPeerCertificate/VerifyConnection; instead, if AIA is enabled at the VU level, we re-wrap the gRPC-local tls.Config so the closure captures gRPC's RootCAs (cp). Added a targeted test that reproduces the failure and now verifies the fix.

• PKCS#7: added a Warn log that detects PKCS#7 responses via Content-Type or the signedData OID, so users can diagnose "AIA didn't work" for those chains. Full parser support tracked as follow-up here AIA: add PKCS#7 parser support for intermediate fetching #6146

Regression check: I also ran the existing TLS scenarios in my sm-k6-tls-test suite against the fixed binary - all still pass, confirming the HTTP AIA path is not regressed.

[szkiba]

LGTM

[mstoykov]

Here if the AIA is enabled we would've already wrtapped it and set the wrapper and InsecureSkipVeriryf.

Above we do copy the latter but not the former but this wrapper funciton checks the latter and doesn't do the wrapping. So this makes gRPC completely insecure.

Please do add a test that checks taht if gRPC certificates do not match and the AIA is enabled it will fail

[joanlopez]

I concur, it's what I tried to explain here: https://github.com/grafana/k6/pull/6137/changes#r3621521110

[vtorosyan]

Thanks both, really good catch!

buildTLSConfig now takes insecureSkipVerify from state.Options.InsecureSkipTLSVerify.Bool (the user's actual intent) instead of parentConfig.InsecureSkipVerify (which the AIA wrapper had flipped to true internally, causing the re-wrap to short-circuit and leave verification off). See 9c0391b

Also added TestBuildTLSConfig_AIARejectsMismatchedCACert covering the scenario you described. I made sure to run the test before and it failed against the pre-fix code and passes now.

[joanlopez]

I think that, when aiaEnabled is on, the VU's tls.Config reaches here with InsecureSkipVerify already set to true by the wrapper, so buildTLSConfig copies that into the fresh config, and the re-wrap sees it and early-returns.

Can you walk through what VerifyPeerCertificate ends up being on the returned config for a gRPC connection that passes cacerts, please?

[joanlopez]

I guess it's not critical, but this allocation could be gated behind:

if tlsAIAFetch := opts.TLSAIAFetch; tlsAIAFetch.Valid && tlsAIAFetch.Bool {
  ...
}

[joanlopez]

LGTM overall, one thing I want to double check though.

In buildVerifyConnFn, we skip verification when cs.ServerName == "", but looks like Go strips that to "" for IP-literal server names, as SNI can't carry IPs.

So for any bare-IP target, this just skips verification entirely.

Go doesn't seem to do that though, it verifies against the raw config.ServerName, and VerifyHostname does check IP literals against the cert's IP SANs, so it's a real check we'd be bypassing here.

Is skipping verification for IP targets on purpose, or should we be checking against the raw ServerName instead of cs.ServerName?
cc/ @vtorosyan @mstoykov

[oleiade]

triage: waiting on @vtorosyan ideas on @joanlopez 's comment above before we move on with merging 🙇🏻

[vtorosyan]

@joanlopez nice catch - I spent some time trying to assess the impact and the best way to move forward. So indeed, returning simply nil is passing the verification which translates to any valid CA-signed cert passing. This is actually a security hole, so we can't leave it like that.

I have tried to properly fix it but it is getting messy... so I am proposing instead just to fail-close for empty ServerName + AIA. In practice the impact is that a user who wants tlsAIAFetch + a bare-IP target will get an error. So they have to either use a hostname, or disable tlsAIAFetch, or set insecureSkipTLSVerify.

I think the impact of this is going to be small, Public CAs generally don't issue certs for IP addresses afaik. So this should be more of a corner case that affects a narrow slice of users.

If you give me your +1 / -1, I will:

1. Commit a change to fail-close
2. Update docs to mention that bare-IP is not supported.

[joanlopez]

@vtorosyan I'm happy with your trade-off, I think it's a good balance 👍🏻 As you said, this will most likely only affect a narrow slice of users, and if that's not the case, we will hear about that, and we can always rework it.

Happy to go with this, let me know once you're done with the actions you mentioned. Thanks! 🙇🏻 🙌🏻

[vtorosyan]

@joanlopez a6dfed7 - thank you!