Switch from Poetry to uv; split server/ into api/, common/, conserver/#137
Open
pavanputhra wants to merge 6 commits intomainfrom
Open
Switch from Poetry to uv; split server/ into api/, common/, conserver/#137pavanputhra wants to merge 6 commits intomainfrom
pavanputhra wants to merge 6 commits intomainfrom
Conversation
- Replace Poetry with uv for 10-100x faster Docker builds - Convert pyproject.toml from Poetry format to PEP 621 + PEP 735 - Add shared storage dependency group (included by both api and conserver) - New Dockerfile.api and Dockerfile.conserver with venv at /opt/venv - Fix all stale server.* imports across 19 production files - Remove dead FastAPI code from conserver/follower.py - Pin deepgram-sdk<4.0.0 and elasticsearch<9.0.0 for API compatibility - Replace poetry.lock with uv.lock - Image size: old monolithic 3.82GB → api 629MB, conserver 1.44GB Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… validation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- 4 parallel build jobs: api + conserver × amd64 + arm64 - amd64 on ubuntu-latest, arm64 on ubuntu-24.04-arm (native, no emulation) - Merge job combines platform digests into multi-arch manifests - Separate GHA cache scopes per service+platform Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates the repo from Poetry to uv and refactors the prior monolithic server/ layout into api/, common/, and conserver/ to produce two slimmer, faster-building Docker images (API + processing pipeline).
Changes:
- Replaces Poetry configuration/lockfile with PEP 621 +
uv.lock, and introduces dependency groups forstorage,api,conserver, anddev. - Adds dedicated Dockerfiles for API and Conserver images with a shared venv at
/opt/venv. - Updates many imports/docs and introduces/ports multiple conserver links, tracers, storage modules, and common test utilities.
Reviewed changes
Copilot reviewed 28 out of 148 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pyproject.toml | Switch to PEP 621 + uv groups |
| docker/Dockerfile.api | New API image build/run config |
| docker/Dockerfile.conserver | New conserver image build/run config |
| conserver/tracers/jlinc/tests/test_jlinc.py | Update tracer test imports |
| conserver/tracers/jlinc/tests/init.py | Add tests package marker |
| conserver/tracers/jlinc/init.py | Add JLINC tracer implementation |
| conserver/tracers/jlinc/README.md | Document JLINC tracer config/usage |
| conserver/links/webhook/init.py | Update import paths |
| conserver/links/webhook/README.md | Add webhook link docs |
| conserver/links/transcribe/init.py | Update import paths |
| conserver/links/tag_router/init.py | Add tag_router link |
| conserver/links/tag_router/README.md | Add tag_router docs |
| conserver/links/tag/init.py | Update import paths |
| conserver/links/tag/README.md | Add tag link docs |
| conserver/links/scitt/readme.md | Add SCITT link docs |
| conserver/links/scitt/dump_cbor.py | Add CBOR debug utility |
| conserver/links/scitt/init.py | Add SCITT link implementation |
| conserver/links/scitt/LICENSE | Add SCITT license text |
| conserver/links/sampler/test_init.py | Add sampler tests |
| conserver/links/sampler/init.py | Add sampler link |
| conserver/links/sampler/README.md | Add sampler docs |
| conserver/links/post_analysis_to_slack/init.py | Update import paths |
| conserver/links/post_analysis_to_slack/README.md | Add Slack link docs |
| conserver/links/openai_transcribe/tests/init.py | Add tests package marker |
| conserver/links/openai_transcribe/init.py | Update import paths |
| conserver/links/jq_link/init.py | Add jq filter link |
| conserver/links/hugging_llm_link/tests/test_llm.py | Add/retain (commented) tests |
| conserver/links/hugging_llm_link/tests/b217b61e-ff68-4f17-a33d-bbc667e8936d.vcon.json | Add test fixture vCon |
| conserver/links/hugging_llm_link/main.py | Add HF LLM link example code |
| conserver/links/hugging_llm_link/init.py | Update import paths |
| conserver/links/hugging_llm_link/README.md | Add HF LLM link docs |
| conserver/links/hugging_face_whisper/tests/test_hugging_face_whisper_integration.py | Add/retain (commented) tests |
| conserver/links/hugging_face_whisper/tests/test_hugging_face_whisper.py | Add/retain (commented) tests |
| conserver/links/hugging_face_whisper/init.py | Update import paths |
| conserver/links/hugging_face_whisper/README.md | Add HF Whisper docs |
| conserver/links/groq_whisper/init.py | Update import paths |
| conserver/links/groq_whisper/README.md | Add Groq Whisper docs |
| conserver/links/expire_vcon/init.py | Add expire_vcon link |
| conserver/links/expire_vcon/README.md | Add expire_vcon docs |
| conserver/links/diet/init.py | Add diet link |
| conserver/links/diet/README.md | Add diet link docs |
| conserver/links/detect_engagement/init.py | Add engagement detection link |
| conserver/links/detect_engagement/README.md | Add detect_engagement docs |
| conserver/links/deepgram_link/init.py | Update import paths |
| conserver/links/deepgram_link/README.md | Add Deepgram docs |
| conserver/links/datatrails/test_datatrails_link.py | Add DataTrails link tests |
| conserver/links/datatrails/LICENSE | Add DataTrails license text |
| conserver/links/check_and_tag/README.md | Add check_and_tag docs |
| conserver/links/analyze_vcon/init.py | Add whole-vCon analysis link |
| conserver/links/analyze_vcon/README.md | Add analyze_vcon docs |
| conserver/links/analyze_and_label/tests/init.py | Add tests package marker |
| conserver/links/analyze_and_label/init.py | Add analyze_and_label link |
| conserver/links/analyze_and_label/README.md | Add analyze_and_label docs |
| conserver/links/analyze/README.md | Add analyze link docs |
| conserver/links/README.md | Add links index documentation |
| conserver/hook.py | Add before/after processing hooks |
| conserver/follower.py | Remove FastAPI usage; update imports |
| common/version.py | Add build-time version helpers |
| common/tests/vcon_fixture.py | Add shared vCon fixture generator |
| common/tests/test_worker_config.py | Add worker/parallel config tests |
| common/tests/test_encoding.py | Add encoding behavior tests |
| common/tests/test_dlq_expiry.py | Add DLQ expiry tests |
| common/tests/test_api.py | Add API lifecycle tests |
| common/tests/logging.conf | Add test logging config |
| common/tests/load_test_vcons.py | Add simple load tool |
| common/tests/lib/filters/test_is_included.py | Add filters tests |
| common/tests/.gitignore | Ignore test artifacts |
| common/storage/vcon_mcp/init.py | Update import paths |
| common/storage/vcon_mcp/README.md | Add vcon-mcp storage docs |
| common/storage/spaceandtime/init.py | Update import paths |
| common/storage/spaceandtime/README.md | Add spaceandtime docs |
| common/storage/sftp/init.py | Update import paths |
| common/storage/sftp/README.md | Add sftp docs |
| common/storage/s3/init.py | Update import paths |
| common/storage/s3/README.md | Add s3 docs |
| common/storage/redis_storage/init.py | Update import paths |
| common/storage/redis_storage/README.md | Add redis storage docs |
| common/storage/postgres/init.py | Update import paths |
| common/storage/postgres/README.md | Add postgres docs |
| common/storage/mongo/init.py | Update import paths |
| common/storage/mongo/README.md | Add mongo docs |
| common/storage/milvus/init.py | Update import paths |
| common/storage/milvus/README.md | Add milvus docs |
| common/storage/file/init.py | Update import paths |
| common/storage/file/README.md | Add file storage docs |
| common/storage/elasticsearch/init.py | Update import paths |
| common/storage/elasticsearch/README.md | Add elasticsearch docs |
| common/storage/dataverse/init.py | Update import paths |
| common/storage/dataverse/README.md | Add dataverse docs |
| common/storage/chatgpt_files/test_init.py | Add ChatGPT files storage tests |
| common/storage/chatgpt_files/init.py | Add ChatGPT files storage impl |
| common/storage/chatgpt_files/README.md | Add ChatGPT files docs |
| common/storage/base.py | Add storage base wrapper |
| common/storage/README.md | Add storage system docs |
| common/settings.py | Add shared settings module |
| common/redis_mgr.py | Add redis client utilities |
| common/logging_dev.conf | Add dev logging config |
| common/logging.conf | Add default logging config |
| common/lib/process_utils.py | Add multiprocessing helpers |
| common/lib/phone_number_utils.py | Add phone util |
| common/lib/metrics.py | Add OTel metrics helpers |
| common/lib/logging_utils.py | Add logger initialization |
| common/lib/listen_list.py | Add async list listener |
| common/lib/links/filters.py | Add link filter helpers |
| common/lib/error_tracking.py | Add Sentry init helpers |
| common/lib/context_utils.py | Add context propagation helpers |
| common/lib/ai_usage.py | Add AI usage tracking helper |
| common/dlq_utils.py | Add DLQ naming helper |
| common/config.py | Add config loading + worker helpers |
| common/init.py | Package marker |
| common/README_CORE.md | Add core architecture docs |
| .gitignore | Ignore redis_data directory |
| .dockerignore | Ignore additional build context dirs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Backport 13 commits from main into the refactored api/common/conserver layout, adapting all file paths and imports from the old server/ structure. Changes ported: - diet link: S3 storage option for dialog bodies with presigned URLs, log redaction for sensitive options (aws_secret_access_key, etc.) - wtf_transcribe link: new link for vfun GPU transcription server, produces WTF-format analysis entries - webhook storage: new post-chain storage backend (parallel to existing conserver/links/webhook chain link) - api: /stats/queue public endpoint for Redis list depth monitoring - api: remove default VCON_REDIS_EXPIRY TTL on ingest; retention now controlled by storage backends - api: index_vcon_parties() extracts party indexing from index_vcon() to avoid redundant Redis reads on ingest path - tests: updated to reflect no-default-TTL and index_vcon_parties changes Also fixes: - pytest.ini pythonpath updated from old 'server' to 'common conserver api' - pyproject.toml dev group: add httpx (required by fastapi TestClient) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
server/into three focused directories:common/— shared lib, storage, settings (used by both images)api/— FastAPI/uvicorn serviceconserver/— processing pipeline and linksstoragedependency group via PEP 735{include-group}(included by bothapiandconserver)docker/Dockerfile.apianddocker/Dockerfile.conserverwith venv at/opt/venvserver.*imports across 19 production filesconserver/follower.pydeepgram-sdk<4.0.0andelasticsearch<9.0.0for API compatibilitypoetry.lockwithuv.locklinux/amd64andlinux/arm64using native runners (no QEMU)Image size improvement
Published images
public.ecr.aws/r4g1k2s3/vcon-dev/vcon-server-api(linux/amd64 + linux/arm64)public.ecr.aws/r4g1k2s3/vcon-dev/vcon-server-conserver(linux/amd64 + linux/arm64)Test plan
🤖 Generated with Claude Code