Skip to content

feat: add kubernetes support#284

Merged
marcelklehr merged 97 commits into
masterfrom
feat/noid/rev-content-flow
Apr 30, 2026
Merged

feat: add kubernetes support#284
marcelklehr merged 97 commits into
masterfrom
feat/noid/rev-content-flow

Conversation

@kyteinsky

@kyteinsky kyteinsky commented Mar 5, 2026

Copy link
Copy Markdown
Contributor

No description provided.

marcelklehr
marcelklehr reviewed Mar 5, 2026
View reviewed changes
Comment thread appinfo/info.xml Outdated
@kyteinsky kyteinsky force-pushed the feat/noid/rev-content-flow branch from 4ca116f to 8e53243 Compare March 5, 2026 10:53
marcelklehr
marcelklehr reviewed Mar 6, 2026
View reviewed changes
Comment thread context_chat_backend/controller.py
marcelklehr
marcelklehr reviewed Mar 10, 2026
View reviewed changes
Comment thread context_chat_backend/controller.py Outdated
@marcelklehr

marcelklehr commented Mar 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Shall we set the context_chat php app to the branch of the corresponding PR here, so we can use CI properly? (temporarily)

@kyteinsky

kyteinsky commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

Shall we set the context_chat php app to the branch of the corresponding PR here, so we can use CI properly? (temporarily)

good idea!

@kyteinsky kyteinsky force-pushed the feat/noid/rev-content-flow branch 2 times, most recently from 83b7587 to 84f6896 Compare March 12, 2026 12:20
kyteinsky and others added 22 commits March 26, 2026 23:02
@kyteinsky
feat: add kubernetes app role selection
5dad714 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
feat: add thread start and stop logic
089d27a 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
wip: migrate the indexing process
64ffdaf 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
wip: parallelize file parsing and processing based on cpu count
03a3f43 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
ci: use the kubernetes branch of context_chat
0dc404b 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix typo
c733982 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
migrate the update process to be thread based
dda312f 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix pydantic types
b09a93c 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: use a dedicated event to allow app halt without app being disabled
11b436c 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix fetch url and pydantic types
c88e153 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: use the correct file id
cd5241e 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: wip: improve embeddings exception handling
4958d1d 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): update to the latest changes
a049121 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): use file to store stderr
795380c 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): add cron jobs
7bc0ed7 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): do a occ files scan before cron jobs
d94c687 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
feat: record indexing errors in content decode function
dadc8fa 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
chore: move file fetch inside injest
f9d86dc 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: truly parallel file parsing and indexing
1ade191 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@marcelklehr @kyteinsky
initial pass at request processing
12fd1ca
@marcelklehr @kyteinsky
implement request processing
8aa2471
@kyteinsky
request processing fixes
2093936 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky

kyteinsky commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

Before we merge, we need to set the context_chat branch back to main

yep, let's see if this passes first

marcelklehr and others added 23 commits April 22, 2026 11:26
@marcelklehr
fix(app_enabled): Add lock for app enabled check
f691743 
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
@kyteinsky
feat: build llama cpp python and add cpu/cuda/vulkan builds
17aa810 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
feat(ci): add kubernetes integration test
10092cb 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: checkout app_api
6819902 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: cache docker build image
2376535 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: correct info.xml path + register command fixes
cf6ba4c 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: use gha as cache backend for docker images
7413e5e 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: replace role names in info.xml
4f86758 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: use local tag so image is not pulled from remote
9ca5dd7 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
chore: show HaRP container's logs
41c85fe 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix: add ghcr.io to the docker image name
cfbc2a1 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@marcelklehr
tests(k8s): make php listen on all interfaces
25ba688 
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
@kyteinsky
fix(ci): use NODE_IP to reach the vector db
3df8fdc 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): increase timeout for context chat stats and handle exit stat…
fd12d84 
…us better

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): checkout the correct branch of app_api
98d2765 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): show all k8s pods' logs
5054791 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): app_api branch translation + only run k8s for master
f74908b 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): separate prompt responses in groups
d3f9575 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(ci): always dump db
c9edba6 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
fix(context): break the loop after the first chunk does not fit in th…
7da999a 
…e context

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
chore(context): increase default context size to 16384
4bbb08c 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@kyteinsky
chore: increase context chunks fetched to 30
205dba7 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
@marcelklehr
feat(ci): add kubernetes integration test (#293)
1c6a929 
- feat: build llama cpp python and add cpu/cuda/vulkan builds
@marcelklehr marcelklehr merged commit 738c149 into master Apr 30, 2026
13 of 14 checks passed
@marcelklehr marcelklehr deleted the feat/noid/rev-content-flow branch April 30, 2026 11:28
@kyteinsky kyteinsky mentioned this pull request May 26, 2026
Merged
kyteinsky added a commit that referenced this pull request May 26, 2026
@kyteinsky
5.4.0-beta0 (#302)
fe0c3a2 
## 5.4.0-beta0 - 2026-05-26

### Added
- add network embedding batching (#276) @fcharlaix-opendsi
- add kubernetes support and reverse content/indexing flow (#284)
@kyteinsky @marcelklehr
- add gh workflows for docker builds and do separate cpu, cuda and rocm
(vulkan) images (#295) @kyteinsky

### Changed
- update readme according to the latest changes (#300) @kyteinsky
- bump llama_cpp_python to 0.3.23 (#301) @kyteinsky

### Fixed
- improve loadSources error handling (#288) @kyteinsky
- fix(pgvector): add chunking to prevent long list of args in queries
(#290) @kyteinsky
- fix(pgvector): make doc deletion query faster (#289) @kyteinsky
@kyteinsky kyteinsky mentioned this pull request Jun 24, 2026
Merged
1 task
kyteinsky added a commit that referenced this pull request Jun 24, 2026
@kyteinsky
5.4.0 (#322)
9c525e5 
## 5.4.0 - 2026-06-24

### Highlights
- The indexing direction has been reversed now. Instead of the
context_chat PHP app sending documents to the context_chat_backend
ExApp, the ExApp downloads the documents from the server according to a
list obtained from the PHP app. This also means that the `occ
context_chat:scan` command serves no purpose and has been removed.
Indexing should be smoother and run continuously now.
- Kubernetes support to scale the CPU computation
- Separate docker images for CPU, CUDA and ROCM (uses Vulkan) instead of
one heavy CUDA/CPU image
- CUDA 12.8 is shipped in the CUDA image so the host drivers should be
updated to this at the minimum.

### Added
- add network embedding batching (#276) @fcharlaix-opendsi
- add kubernetes support and reverse content/indexing flow (#284)
@kyteinsky @marcelklehr
- add gh workflows for docker builds and do separate cpu, cuda and rocm
(vulkan) images (#295) @kyteinsky

### Changed
- update readme according to the latest changes (#300) @kyteinsky
- bump llama_cpp_python to 0.3.23 (#301) @kyteinsky
- move task types to the backend (#321) @kyteinsky
- adjust comment in Dockerfile regarding RTX5090 support (#316)
@kyteinsky

### Fixed
- improve loadSources error handling (#288) @kyteinsky
- fix(pgvector): add chunking to prevent long list of args in queries
(#290) @kyteinsky
- fix(pgvector): make doc deletion query faster (#289) @kyteinsky
- drop latin-1 decode in source title and userIds (#306) @kyteinsky
- handle validation errors of files and content providers individually
(#308) @kyteinsky
- prevent race condition in vectordb tables creation (#308) @kyteinsky
- pass actual error in the error object (#308) @kyteinsky
- add container hostname to /etc/hosts to silence sudo warning (#311)
@sanzakicesarr

## 🤖 AI (if applicable)

- [ ] The content of this PR was partly or fully generated using AI

Signed-off-by: kyteinsky <kyteinsky@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@marcelklehr marcelklehr marcelklehr approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kyteinsky @marcelklehr