Skip to content

Panic (SIGSEGV / nil pointer) in gzipResponseWriter.Flush when serving a gzipped batch RPC request #22334

Description

@ctrl-mx

Summary

When the RPC server serves a request that (a) has Accept-Encoding: gzip and (b) triggers a Flush on the response before any body byte has been written to the gzip writer, the node panics with a nil pointer dereference inside compress/flate.(*Writer).Flush. The panic is
fatal: the process exits (exited with code 2) and is restarted by the supervisor. Because the crash is driven by an incoming request pattern, it becomes a crash loop — every such request kills the node again.

The node runs fine indefinitely until a client starts sending the triggering request shape, so this is a latent bug rather than a runtime degradation.

Environment

  • Erigon version: 3.5.0
  • OS / deployment: Linux, Docker container (erigon-1)
  • Reverse proxy: HAProxy in front of the RPC endpoint (127.0.0.1:8545), mode http
  • Client sends JSON-RPC batch requests with Accept-Encoding: gzip

Stack trace

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7b9021]
 
goroutine 1683252 [running]:
compress/flate.(*Writer).Flush(...)
        compress/flate/deflate.go:722
compress/gzip.(*Writer).Flush(0xc29479ab40)
        compress/gzip/gzip.go:221 +0x61
github.com/erigontech/erigon/node.(*gzipResponseWriter).Flush(0xc28bb5b920)
        github.com/erigontech/erigon/node/rpcstack.go:608 +0x225
github.com/erigontech/erigon/rpc.(*handler).runMethod(0x60?, {0x3d16740, 0xc2947051d0}, 0xc25c1b2a80, 0xc04d0801b0, {0xc28bb5bcb0, 0x0, 0xc2e3f42c20})
        github.com/erigontech/erigon/rpc/handler.go:693 +0x87
github.com/erigontech/erigon/rpc.(*handler).handleCall(0xc2701ebc70, 0xc28bb5bc20, 0xc25c1b2a80, {0x3d5ba30, 0xc2e3f42c20})
        github.com/erigontech/erigon/rpc/handler.go:583 +0x2ee
github.com/erigontech/erigon/rpc.(*handler).handleCallMsg(0xc2701ebc70, 0xc28bb5bc20, 0xc25c1b2a80, {0x3d5ba30, 0xc2e3f42c20})
        github.com/erigontech/erigon/rpc/handler.go:524 +0x39a
github.com/erigontech/erigon/rpc.(*handler).handleBatch.func4.1(0x1)
        github.com/erigontech/erigon/rpc/handler.go:228 +0x245
github.com/erigontech/erigon/rpc.(*handler).handleBatch.func4 in goroutine 1683250
        github.com/erigontech/erigon/rpc/handler.go:210 +0x105
erigon-1 exited with code 2 (restarting)

Steps to reproduce

  1. Run an Erigon 3.5.0 node with the HTTP RPC endpoint enabled and HTTP compression on (default).
  2. Send a JSON-RPC batch request to the endpoint with header Accept-Encoding: gzip.
  3. The node panics and restarts. From the client side this appears as 502 for the request that killed the node, followed by 503 for subsequent requests while the node is restarting and marked down by the proxy health check.
    Notes on reproduction:
  • The crash happens within a few seconds of the client starting, well below any proxy timeout, so it is not timeout-related.
  • With the client stopped, the node is stable.
  • The panic goes through handleBatch, so a batch (rather than a single call) appears to be part of the trigger.

Expected behavior

Serving an RPC response — including an empty/streaming/batch response — with gzip enabled should never panic. A Flush on a gzip response writer that has not yet written any body should be a no-op (or safely initialize the writer), not dereference a nil compressor.

Actual behavior

gzipResponseWriter.Flush (node/rpcstack.go:608) calls into gzip.(*Writer).Flush, which calls flate.(*Writer).Flush, dereferencing a nil internal flate compressor. compress/gzip initializes its underlying flate.Writer lazily on the first Write; calling Flush before any Write therefore dereferences nil and crashes the whole process.

Suspected root cause

gzipResponseWriter.Flush calls Flush on the wrapped gzip.Writer unconditionally, even when nothing has been written yet. In compress/gzip, the flate.Writer is created lazily on the first Write, so an early Flush (empty body, streaming response, or a sub-response in a batch that flushes before writing) hits a nil compressor.

A guard in gzipResponseWriter.Flush that skips the flush when no bytes have been written (or that ensures the gzip writer is initialized before flushing) should resolve it.

Workarounds (confirmed)

  • At the reverse proxy (confirmed to fix it): strip the request's encoding header so the node never enters the gzip path. In HAProxy:
  http-request del-header Accept-Encoding

After applying this, the crashes stopped completely.

  • At the node: disabling HTTP compression via --http.compression=false (and --ws.compression=false for WebSocket) should have the same effect by removing the gzip response path.
    Increasing proxy timeouts did not help, confirming the crash is not timeout-driven.

Additional context

The memory stats line printed just before the panic
(alloc=12.3GB sys=17.9GB) is normal Go runtime memory usage for the node and is unrelated to the crash.

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions