docs(spider): Remove legacy Spider integration doc. - #2436
Conversation
WalkthroughSpider services and deployment variants were removed from deployment architecture documentation, Docker Compose startup instructions, and user documentation navigation. The dedicated Spider usage guide was also deleted. ChangesSpider documentation removal
Estimated code review effort: 1 (Trivial) | ~5 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
We do have a docker-compose file for the legacy spider setup. We don't need to remove it in this PR, but we should probably make an issue to keep track of it for removal.
| ### Deployment types | ||
|
|
||
| CLP supports multiple deployment configurations based on the compression scheduler and query engine. | ||
| CLP supports multiple deployment configurations based on the query engine. | ||
|
|
||
| | Deployment Type | Compression Scheduler | Query Engine | | ||
| |-----------------|-----------------------|------------------------------| | ||
| | Base | Celery | [Presto][presto-integration] | | ||
| | Full | Celery | Native | | ||
| | Spider Base | Spider | [Presto][presto-integration] | | ||
| | Spider Full | Spider | Native | | ||
|
|
||
| :::{note} | ||
| Spider support is not yet available for Helm. | ||
| ::: | ||
|
|
||
| Docker Compose selects the appropriate compose file (e.g., `docker-compose.yaml` for Full, | ||
| `docker-compose-spider.yaml` for Spider Full) and uses `deploy.replicas` with environment | ||
| variables (e.g., `CLP_MCP_SERVER_ENABLED`) to toggle optional services. Helm uses conditional | ||
| templating to include/exclude resources. | ||
| Docker Compose uses `deploy.replicas` with environment variables (e.g., `CLP_MCP_SERVER_ENABLED`) | ||
| to toggle optional services. Helm uses conditional templating to include or exclude resources. |
There was a problem hiding this comment.
Since there's only one type of compression scheduler now, shall we remove the "Compression Scheduler" column?
Depending on how we're gonna add the new Spider stuff, I will reword the table in a later PR.
There was a problem hiding this comment.
Since there's only one type of compression scheduler now, shall we remove the "Compression Scheduler" column?
i agree.
it might be easier to read today by turning the table into a simple list. e.g.,
CLP supports two deployment configurations, depending on the query engine:
- Full: Uses Celery for query orchestration.
- Base: Excludes the Celery-based orchestration in Full, to support the [Presto integration][presto-integration].
| @@ -251,23 +241,15 @@ Services require persistent storage for logs, data, archives, and streams. | |||
|
|
|||
| ### Deployment types | |||
There was a problem hiding this comment.
Changes in this section may require a sanity check from @junhaoliao.
There was a problem hiding this comment.
There was a problem hiding this comment.
Directly modified the PR title to include (spider).
Co-authored-by: Lin Zhihao <59785146+LinZhihao-723@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/src/dev-docs/design-deployment-orchestration.md`:
- Around line 244-248: Decouple the documented Base/Full deployment models from
the shipped Compose stack: update the Compose configuration to remove or disable
spider-scheduler and spider-compression-worker, or explicitly document the
required Spider-specific Compose profile alongside the Helm deployment model.
Ensure the Base path no longer exposes unsupported Spider services by default.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e34d9d1e-aafc-496c-ace8-978c26bb5fc6
📒 Files selected for processing (1)
docs/src/dev-docs/design-deployment-orchestration.md
Description
This PR removes documentation for the legacy Spider integration in preparation for documenting the new Helm-based integration by:
Checklist
breaking change.
Validation performed
task docs:sitebuilt successfully.spiderandSpidershowed no result.Summary by CodeRabbit