Skip to content

Commit

Permalink
Fix session_id bug (#205)
Browse files Browse the repository at this point in the history 
* Replace PaiEas LLM with LLI-integration and upgrade python to 3.11 (#148)

* Replace PaiEas LLM with LLI-integration and upgrade python version to 3.11

* Replace MyFCDashScope with OpenAILike class

* Fix pyproject dependency

* bug fix (#149)

* Support postgresql load user dict (#150)

* make format

* Allow not install extension pg_jieba

* table name data_default

* Convert raptor processor to TransformComponent (#151)

* udpate raptor using transform

* modify raptor with transform

* modify raptor and dataloader

---------

Co-authored-by: Yue Fei <luxun.fy@alibaba-inc.com>

* Add clip model (#130)

* Update

* Add clip model

* Fix oss cache

* Fix cache

* Pdf reader upload image

* Add multimodal

* Update config

* Use two embedding

* Add text_image node

* Add tests

* Fix tests

* fix multi_modal_vector

---------

Co-authored-by: 燃夏 <chenanyu.cay@alibaba-inc.com>

* Fix docker base image (#152)

* change insert to be sync (#153)

* Personal/ranxia/fix image readme (#155)

* fix multi_modal and readme

* fix multi_modal and readme

* fix multi_modal and readme

* fix multi_modal image (#156)

* Support Agentic RAG with intent and functioncalling (#154)

* Add intent detection module

* Remove LlmQuery class

* Support API

* Refactor agent module and format toml

* Refactor module tool

* Refactor query api

* Add demo and UI

* remove

* Fix reviews

* Add test for intent and api

* Add web search (#161)

* Add web search

* Fix lint

* Fix bug

* Update timeout

* Fix bug

* Fix jieba bug (#163)

* Support PAI-EAS MultiModal LLM (#168)

* Support minicpm

* Fix issue

* Bugfix: PaiEas LLM endpoint & max_tokens (#171)

* Fix dashscope interface (#172)

* Fix dashscope llm

* Fix bug

* Fix test bug (#174)

* add minerU (#160)

* add minerU

* add minerU

* add minerU

* Fix nodes id and simi_topK

* remove image url from text

* remove image url from text

* remove image url from text

* Support FAQ query w/o image (#162)

* Support FAQ query w/o image

* Using LLM when query w/o images

* Personal/ranxia/mineru enhancement (#164)

* remove repeat nodes

* show multiple pictures in media

* show multiple pictures in media

* Install miner with poetry (#165)

* fix retriever

* Support OSS Data Loader (#166)

* Support oss data loader

* Skip file which has been uploaded

* Support oss prefix via api

* 1. change image size (#167)

2. limit image number
3. fix retriever answer ui format

* adjust image score (#169)

* merge feature

* merge feature

* merge feature

* merge feature

* Fix bug (#173)

* Support chunk text-overflow display (#170)

* Fix bugs

* Support text-overflow

* Support text-overflow

* Support load MinerU config file automatically (#175)

* Support load MinerU config file automatically

* Modify

* Direct writing the config rather than copying

* Fix multi_modal build docker (#176)

* fix load_config (#177)

* change  multimodal prompt (#178)

* Test Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix test bug (#174) (#179)

Co-authored-by: Yue Fei <luxun.fy@alibaba-inc.com>

* Fix Dockerfile (#180)

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix Dockerfile

* Fix docker env (#181)

* Fix Dockerfile

* Fix bugs

* Fix docker env

* Fix docker env

* Fix docker env (#183)

* Fix Dockerfile

* Fix bugs

* Fix docker env

* Fix docker env

* Fix docker env

* Fix docker env

* Fix docker env

* Bugfix

* Bugfix for EAS (#184)

* Fix Dockerfile

* Fix bugs

* Fix docker env

* Fix docker env

* Fix docker env

* Fix docker env

* Fix docker env

* Bugfix

* Bugfix

* Fix detectron link (#182)

* Update detectron dependency (#185)

* Update dependency

* udpate poetry lock

* fix multimodal_config and prompt (#186)

* fix MinerU readme (#189)

* Add timeout and more logs (#188)

* Personal/ranxia/fix miner u readme (#190)

* fix MinerU readme

* fix MinerU readme

* Personal/ranxia/fix miner u readme (#191)

* fix MinerU readme

* fix MinerU readme

* fix MinerU config

* fix MinerU bug (#192)

* Personal/ranxia/fix test and review bug (#193)

* fix MinerU bug

* fix MinerU bug

* fix MinerU bug

* fix MinerU bug

* fix MinerU bug

* fix MinerU bug

* fix MinerU bug

---------

Co-authored-by: 筱文 <zxw320697@alibaba-inc.com>
Co-authored-by: Yue Fei <luxun.fy@alibaba-inc.com>

* fix multimodal readme and config (#195)

* nl2sql refactoring (#194)

* change insert to be sync

* add nl2sql

* nl2sql setting

* nl2sql setting

* fix test bug

* fix bugs

* data analysis retriever and synthesizer

* fix tests bugs

* add data_analysis ui

* update poetry.lock

* remove unnecessary comment

* add fault tolerance if no file provided

* add minor fault tolerance

* add upload_datasheet

* nl2sql refactor and add db ui

* restore retriever & synthesizer

* update poetry.lock

* Fix list merge

* bug fix

* add default display

---------

Co-authored-by: 陆逊 <luxun.fy@alibaba-inc.com>

* Personal/xi/nl2sql UI (#196)

* change insert to be sync

* add nl2sql

* nl2sql setting

* nl2sql setting

* fix test bug

* fix bugs

* data analysis retriever and synthesizer

* fix tests bugs

* add data_analysis ui

* update poetry.lock

* remove unnecessary comment

* add fault tolerance if no file provided

* add minor fault tolerance

* add upload_datasheet

* nl2sql refactor and add db ui

* restore retriever & synthesizer

* update poetry.lock

* Fix list merge

* bug fix

* add default display

* data_analysis ui update

---------

Co-authored-by: 陆逊 <luxun.fy@alibaba-inc.com>

* Personal/ranxia/change max new tokens (#199)

* set multimodal llm max_new_tokens

* set multimodal llm max_new_tokens

* Add trace (#197)

* Add trace

* Fix bug

* Push to hangzhou region by default

* 修复tables和descriptions默认配置bug (#198)

* change insert to be sync

* add nl2sql

* nl2sql setting

* nl2sql setting

* fix test bug

* fix bugs

* data analysis retriever and synthesizer

* fix tests bugs

* add data_analysis ui

* update poetry.lock

* remove unnecessary comment

* add fault tolerance if no file provided

* add minor fault tolerance

* add upload_datasheet

* nl2sql refactor and add db ui

* restore retriever & synthesizer

* update poetry.lock

* Fix list merge

* bug fix

* add default display

* data_analysis ui update

* fix table & description & query_output bugs

* fix inconsistency between frontend and backend data structures

---------

Co-authored-by: 陆逊 <luxun.fy@alibaba-inc.com>

* Fix nginx routing (#200)

* Fix nginx routing (#202)

* Fix nginx routing

* Fix nginx config

* add data_analysis doc (#201)

Co-authored-by: Yue Fei <luxun.fy@alibaba-inc.com>

* Resolve conflict

* Fix session_id bug (#204)

---------

Co-authored-by: wwxxzz <zxw320697@alibaba-inc.com>
Co-authored-by: aero-xi <chuyu.cx@alibaba-inc.com>
Co-authored-by: zt2645802240 <47960912+zt2645802240@users.noreply.github.com>
Co-authored-by: 燃夏 <chenanyu.cay@alibaba-inc.com>
  • Loading branch information
@moria97 @wwxxzz
@aero-xi @zt2645802240 @Ceceliachenen
5 people authored Sep 6, 2024
1 parent dac813e commit 005a003
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 16 deletions.
16 changes: 4 additions & 12 deletions src/pai_rag/app/web/rag_client.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -198,18 +198,14 @@ def query_search(
raise RagApiError(code=r.status_code, msg=r.text)
if not stream:
response = dotdict(json.loads(r.text))
yield self._format_rag_response(
text, response, session_id=session_id, stream=stream
)
yield self._format_rag_response(text, response, stream=stream)
else:
full_content = ""
for chunk in r.iter_lines(chunk_size=8192, decode_unicode=True):
chunk_response = dotdict(json.loads(chunk))
full_content += chunk_response.delta
chunk_response.delta = full_content
yield self._format_rag_response(
text, chunk_response, session_id=session_id, stream=stream
)
yield self._format_rag_response(text, chunk_response, stream=stream)

def query_data_analysis(
self,
Expand All @@ -228,18 +224,14 @@ def query_data_analysis(
raise RagApiError(code=r.status_code, msg=r.text)
if not stream:
response = dotdict(json.loads(r.text))
yield self._format_rag_response(
text, response, session_id=session_id, stream=stream
)
yield self._format_rag_response(text, response, stream=stream)
else:
full_content = ""
for chunk in r.iter_lines(chunk_size=8192, decode_unicode=True):
chunk_response = dotdict(json.loads(chunk))
full_content += chunk_response.delta
chunk_response.delta = full_content
yield self._format_rag_response(
text, chunk_response, session_id=session_id, stream=stream
)
yield self._format_rag_response(text, chunk_response, stream=stream)

def query_llm(
self,
Expand Down
3 changes: 1 addition & 2 deletions src/pai_rag/app/web/tabs/agent_tab.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,8 @@ def respond(agent_question, agent_chatbot):


def clear_history(chatbot):
rag_client.clear_history()
chatbot = []
global current_session_id
current_session_id = None
return chatbot


Expand Down
3 changes: 1 addition & 2 deletions src/pai_rag/app/web/tabs/data_analysis_tab.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,8 @@ def analysis_respond(question, chatbot):


def clear_history(chatbot):
rag_client.clear_history()
chatbot = []
global current_session_id
current_session_id = None
return chatbot


Expand Down

0 comments on commit 005a003

Please sign in to comment.