feat(router): Hive Console Usage Reporting

[ardatan]

Hive Console Client integration
Ref ROUTER-102
Blocked by graphql-hive/console#7143

Documentation -> graphql-hive/console#7171

TODOs:

• Release hive-console-sdk and add it to Cargo.toml here
• Update documentation and env overrides after feat(config): configuration override using env vars, enable/disable graphiql via config #519 lands.

[gemini-code-assist]

Summary of Changes

Hello @ardatan, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant new feature: the ability for the GraphQL router to report detailed usage metrics to the GraphQL Hive Console. This integration provides valuable insights into how the router is being used, including operation names, execution times, and error rates. The reporting mechanism is highly configurable, allowing users to control aspects like sampling rates, excluded operations, and reporting intervals, ensuring efficient and relevant data collection. This enhancement is crucial for monitoring and optimizing GraphQL API performance and adoption.

Highlights

• Hive Console Usage Reporting Integration: The router now integrates with the hive-console-client to report GraphQL operation usage data, providing insights into router activity.
• Configurable Usage Settings: A new UsageConfig struct has been introduced, allowing detailed configuration of reporting parameters such as API token, endpoint, sample rate, excluded operations, client headers, buffer size, and various timeouts.
• Dynamic Usage Agent Initialization and Background Task: A UsageAgent is initialized based on the provided UsageConfig and registered as a background task, ensuring that usage reports are periodically flushed to the Hive Console.
• GraphQL Pipeline Integration: The GraphQL execution pipeline has been modified to capture operation details, execution duration, and error counts, which are then conditionally sent to the UsageAgent for reporting.
• Dependency Updates and Forked Parser: The project's dependencies have been updated, notably replacing the graphql-parser crate with graphql-parser-hive-fork across Cargo.lock and Cargo.toml files, and adding new crates like md5 and webpki-roots.
• Error Counting in Execution Output: The PlanExecutionOutput now includes an error_count field, which tracks the number of errors encountered during query plan execution, providing crucial data for usage reporting.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces Hive Console Client integration for usage reporting in the Hive Router. It includes changes to Cargo.lock and Cargo.toml files to add the new dependency, modifications to bin/router/src/lib.rs and bin/router/src/pipeline/mod.rs to implement the usage reporting logic, and a new file bin/router/src/pipeline/usage.rs for sending usage reports. The shared state is also updated to include the usage agent. I have provided review comments to address potential issues related to error handling and code clarity.

[github-actions]

🐋 This PR was built and pushed to the following Docker images:

Image Names: ghcr.io/graphql-hive/router

Platforms: linux/amd64,linux/arm64

Image Tags: ghcr.io/graphql-hive/router:pr-499 ghcr.io/graphql-hive/router:sha-f37f916

Docker metadata

{
"buildx.build.ref": "builder-3120460a-6daa-42e0-a588-a96b5a30c41a/builder-3120460a-6daa-42e0-a588-a96b5a30c41a0/zxdo7heuqk99hrrp7r3fqyph5",
"containerimage.descriptor": {
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "digest": "sha256:543ce52f3909f85c94da6848507319c6eae02efe51b57a5175d25e84a0a82f22",
  "size": 1609
},
"containerimage.digest": "sha256:543ce52f3909f85c94da6848507319c6eae02efe51b57a5175d25e84a0a82f22",
"image.name": "ghcr.io/graphql-hive/router:pr-499,ghcr.io/graphql-hive/router:sha-f37f916"
}

[github-actions]

✅ k6-benchmark results

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 219447      ✗ 0    
     data_received..................: 6.4 GB  214 MB/s
     data_sent......................: 86 MB   2.9 MB/s
     http_req_blocked...............: avg=2.61µs   min=711ns   med=1.69µs  max=3.61ms   p(90)=2.48µs  p(95)=2.89µs  
     http_req_connecting............: avg=142ns    min=0s      med=0s      max=1.45ms   p(90)=0s      p(95)=0s      
     http_req_duration..............: avg=20.09ms  min=2.14ms  med=19.12ms max=151.53ms p(90)=27.4ms  p(95)=30.47ms 
       { expected_response:true }...: avg=20.09ms  min=2.14ms  med=19.12ms max=151.53ms p(90)=27.4ms  p(95)=30.47ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 73169
     http_req_receiving.............: avg=145.57µs min=28.02µs med=43.83µs max=56.97ms  p(90)=82.34µs p(95)=392.14µs
     http_req_sending...............: avg=20.04µs  min=4.31µs  med=9.97µs  max=17.53ms  p(90)=14.88µs p(95)=21.1µs  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s      p(95)=0s      
     http_req_waiting...............: avg=19.92ms  min=2.08ms  med=18.99ms max=120.02ms p(90)=27.15ms p(95)=30.14ms 
     http_reqs......................: 73169   2433.598212/s
     iteration_duration.............: avg=20.5ms   min=5.82ms  med=19.43ms max=255.08ms p(90)=27.8ms  p(95)=30.87ms 
     iterations.....................: 73149   2432.933012/s
     vus............................: 50      min=50        max=50 
     vus_max........................: 50      min=50        max=50 

[dotansimha]

One more thing, I think we might need to expose these by default using env vars. See env_var_override for example on how to do it. We can align to how it looks like in GW.

[github-actions]

🐋 This PR was built and pushed to the following Docker images:

Image Names: ghcr.io/graphql-hive/router

Platforms: linux/amd64,linux/arm64

Image Tags: ghcr.io/graphql-hive/router:pr-499 ghcr.io/graphql-hive/router:sha-40044d1

Docker metadata

{
"buildx.build.ref": "builder-b67a8746-4b4d-4dc9-95aa-b3c663d234a1/builder-b67a8746-4b4d-4dc9-95aa-b3c663d234a10/rldm6ekk6lczq2k5xfsvkzve2",
"containerimage.descriptor": {
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "digest": "sha256:9baabe45f243ec1f3251bc4ef76f66073ca24a64a5b0ce9367aa42fa319fda0b",
  "size": 1609
},
"containerimage.digest": "sha256:9baabe45f243ec1f3251bc4ef76f66073ca24a64a5b0ce9367aa42fa319fda0b",
"image.name": "ghcr.io/graphql-hive/router:pr-499,ghcr.io/graphql-hive/router:sha-40044d1"
}