-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add extensible compression support for RPC requests #6084
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
a2eba9c to
4917aef
Compare
4fb2574 to
b8939aa
Compare
8dc6615 to
acb0979
Compare
|
@lutter This is also building now ... |
8bab643 to
cf7477b
Compare
|
Rebased to latest master (and cleaned up a little) |
|
I just looked at this, and seeing how we do similar things for other providers, why do we use a separate |
|
This pull request hasn't had any activity for the last 90 days. If there's no more activity over the course of the next 14 days, it will automatically be closed. |
- Replace boolean compression_enabled with Compression enum (None, Gzip) - Support per-provider compression configuration via "compression" field - Add placeholders for future compression methods (Brotli, Deflate) - Update transport layer to handle compression enum with match statement - Add comprehensive unit tests for compression configuration parsing - Update example configuration and documentation Configuration examples: compression = "gzip" # Enable gzip compression compression = "none" # Disable compression (default) Addresses issue #5671 with future-extensible design. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
When reqwest gzip feature is enabled globally, Client::default() enables automatic gzip compression which removes content-length headers from responses. ArweaveClient needs content-length to check file sizes, so explicitly disable gzip for Arweave requests. Fixes test: polling_monitor::ipfs_service::test::arweave_get 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
b7f93ce to
e201a75
Compare
@lutter the PR has been refactored and the branch rebased. The format is now NOTE: The PR went stale and automatically closed due to no activity for months. It would be good to have this reviewed and merged into master. |
e201a75 to
814a9fb
Compare
920cb28 to
339e484
Compare
339e484 to
8e01d4c
Compare
Summary
Adds configurable compression support for Graph Node's outgoing JSON-RPC requests to upstream providers, supporting gzip, brotli, and deflate methods.
Goal
Allow node operators to enable compression on a per-provider basis for outgoing RPC requests, reducing bandwidth usage between Graph Node and upstream providers.
Design
Compression is configured through the existing features array on providers using
compression/gzip,compression/brotli, orcompression/deflate. ACompressionenum is used internally to represent the selected method. At most onecompression/*feature may be set per provider — this is validated at config load time.Changes
Transport layer (
chain/ethereum/src/transport.rs)Compressionenum withNone(default),Gzip,Brotli,DeflatevariantsTransport::new_rpc()accepts aCompressionparameter and calls the appropriate reqwest builder methodConfig layer (
node/src/config.rs)compression/gzip,compression/brotli, andcompression/deflatetoPROVIDER_FEATUREScompression() -> CompressiononWeb3Providerthat maps features to the enumcompression/*feature is present per providerWiring layer (
node/src/chain.rs)web3.compression()and passes it toTransport::new_rpc()Network tests (
chain/ethereum/src/network.rs)Transport::new_rpc()test call sites to useCompression::NonePublic exports (
chain/ethereum/src/lib.rs)CompressionalongsideTransportDependencies (
graph/Cargo.toml)"gzip","brotli", and"deflate"to reqwest featuresArweaveClient fix (
graph/src/components/link_resolver/arweave.rs)gzipfeature makesClient::default()automatically negotiate gzip, which stripscontent-lengthheaders that ArweaveClient depends on for file size checks.Configuration
Omitting any
compression/*feature means no compression (the default).Full example
Test plan
cargo check --releasepassesjust lintpasses with zero warnings