Add background vector store indexing and proper HTTP 503 handling#75
Open
Sasyamerugu wants to merge 1 commit intosugarlabs:mainfrom
Open
Add background vector store indexing and proper HTTP 503 handling#75Sasyamerugu wants to merge 1 commit intosugarlabs:mainfrom
Sasyamerugu wants to merge 1 commit intosugarlabs:mainfrom
Conversation
… initialization Previously the vector store was built during server startup using agent.setup_vectorstore(settings.DOC_PATHS). This blocks the API startup when documents are large and prevents the server from serving requests until indexing is finished. This change moves vector store construction into a background thread, allowing the API to start immediately while indexing runs asynchronously. The /ask endpoint now checks whether the retriever has been initialized. If the vector store is still building, the endpoint returns HTTP 503 (Service Unavailable) indicating the service is temporarily unavailable. Other endpoints such as /ask-llm and /ask-llm-prompted are unaffected, as they do not depend on the vector store and can serve requests normally while indexing is in progress. Additionally, a bug was fixed where HTTPException responses (such as 503) were accidentally converted to HTTP 500 because they were caught by a generic exception handler. The handler now re-raises HTTPException to preserve the correct response status.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously the vector store was built during server startup using agent.setup_vectorstore(settings.DOC_PATHS). This blocks the API startup when documents are large and prevents the server from serving requests until indexing is finished.
This change moves vector store construction into a background thread, allowing the API to start immediately while indexing runs asynchronously.
The /ask endpoint now checks whether the retriever has been initialized. If the vector store is still building, the endpoint returns HTTP 503 (Service Unavailable) indicating the service is temporarily unavailable.
Other endpoints such as /ask-llm and /ask-llm-prompted are unaffected, as they do not depend on the vector store and can serve requests normally while indexing is in progress.
Additionally, a bug was fixed where HTTPException responses (such as 503) were accidentally converted to HTTP 500 because they were caught by a generic exception handler. The handler now re-raises HTTPException to preserve the correct response status.