sequenceDiagram
participant GoldenDataset as Golden Dataset (DB/File)
participant EvalScript as Evaluation Script (NestJS)
participant ChunkingSvc as Semantic Chunking Service (TS)
participant EmbeddingSvc as Embedding Service (Python ECS/Lambda)
participant Neo4j as Neo4j Module
participant BaseLLM as Base LLM API (e.g., GPT-4)
participant FineTunedLLM as Fine-tuned LLM (Vertex AI)
participant Prisma as Prisma Client
EvalScript->>GoldenDataset: Load Chat & Golden Tags
GoldenDataset-->>EvalScript: Return Chat Data, Golden Tag IDs
EvalScript->>ChunkingSvc: Process Chat Messages
ChunkingSvc-->>EvalScript: Return Semantic Chunks
EvalScript->>EmbeddingSvc: Generate Embeddings for Chunks
EmbeddingSvc-->>EvalScript: Return Embeddings
%% Graph Path Evaluation %%
EvalScript->>Neo4j: Store Chunk & Embedding (if not exists)
EvalScript->>Neo4j: Find Similar Chunks/Tags (Vector Search)
Neo4j-->>EvalScript: Return Potential Graph Tags & Scores
%% Base LLM Path Evaluation %%
EvalScript->>BaseLLM: Send Chunk Text + Prompt for Tags
BaseLLM-->>EvalScript: Return Potential Base LLM Tags & Scores
%% Fine-tuned LLM Path Evaluation %%
EvalScript->>FineTunedLLM: Send Chunk Text + Prompt for Tags
FineTunedLLM-->>EvalScript: Return Potential Fine-tuned Tags & Scores
%% Comparison %%
EvalScript->>EvalScript: Compare Graph Tags vs Golden Tags
EvalScript->>EvalScript: Compare Base LLM Tags vs Golden Tags
EvalScript->>EvalScript: Compare Fine-tuned Tags vs Golden Tags
EvalScript->>EvalScript: Calculate Accuracy, Latency, etc.
EvalScript->>Prisma: Store Evaluation Results (Optional)
Explanation: The evaluation script takes a chat from the golden dataset, processes it (chunking, embedding), and then sends it to each tagging method (Graph DB search, Base LLM API, Fine-tuned LLM API). The results from each method are compared against the known "golden" tags to calculate accuracy and other metrics, informing the decision in Scope 2.
sequenceDiagram
participant DynamoStream as DynamoDB Stream
participant IngestionSvc as Ingestion Service (Lambda/NestJS)
participant ChunkingSvc as Semantic Chunking Service (TS)
participant EmbeddingSvc as Embedding Service (Python ECS/Lambda)
participant TaggingOrch as Tagging Orchestration (NestJS)
participant Neo4j as Neo4j Module
participant FineTunedLLM as Fine-tuned LLM (Vertex AI)
participant Prisma as Prisma Client
DynamoStream->>IngestionSvc: New Chat Message Event(s)
IngestionSvc->>ChunkingSvc: Process Recent Messages for Chat
ChunkingSvc-->>IngestionSvc: Return Updated/New Chunk(s)
alt New/Updated Chunk Created
IngestionSvc->>EmbeddingSvc: Generate Embedding for Chunk
EmbeddingSvc-->>IngestionSvc: Return Embedding
IngestionSvc->>TaggingOrch: Trigger Tagging for Chat (with Chunk/Embedding)
TaggingOrch->>Neo4j: Store Chunk & Embedding
TaggingOrch->>Neo4j: Find Tags via Vector Search & Communities
Neo4j-->>TaggingOrch: Return Graph Tags & Confidence
alt Graph Tags Sufficient (e.g., >= high confidence)
TaggingOrch->>Prisma: Create/Update TagsOnChats record(s) (Source: 'auto-graph', Confidence)
Prisma-->>TaggingOrch: Confirm Write
else Graph Tags Insufficient
TaggingOrch->>FineTunedLLM: Predict Tags for Chunk Text
FineTunedLLM-->>TaggingOrch: Return LLM Tags & Confidence
TaggingOrch->>TaggingOrch: Combine Graph & LLM Tags (Apply Hybrid Logic)
TaggingOrch->>Prisma: Create/Update TagsOnChats record(s) (Source: 'auto-hybrid', Confidence)
Prisma-->>TaggingOrch: Confirm Write
end
end
Explanation: New messages trigger ingestion. They are chunked and embedded. The Tagging Orchestrator first queries Neo4j. If the results are good enough, tags are saved via Prisma. If not, it calls the fine-tuned LLM on Vertex AI, combines the results, and then saves the final tag assignments to `TagsOnChats`.
sequenceDiagram
participant User as Clinic Staff User
participant UI as Manual Tagging UI
participant API as NestJS API (App Runner)
participant Prisma as Prisma Client
participant ChatTag as ChatTag Table
participant TagsOnChats as TagsOnChats Table
User->>UI: Selects Chat, Clicks 'Add Tag'
UI->>API: POST /chats/{chatId}/tags (body: { tagId: "tag-uuid-123" })
API->>Prisma: findUnique ChatTag where id=tagId
Prisma-->>API: Return ChatTag details
API->>Prisma: create TagsOnChats record (chatId, tagId, source: 'manual')
Prisma->>TagsOnChats: Insert Row
TagsOnChats-->>Prisma: Confirm Insert
Prisma-->>API: Confirm Create
API-->>UI: Success Response (201 Created)
UI->>User: Show Tag Added to Chat
User->>UI: Clicks 'Remove Tag' on an existing tag
UI->>API: DELETE /chats/{chatId}/tags/{tagId}
API->>Prisma: delete TagsOnChats record where chatId=chatId AND tagId=tagId
Prisma->>TagsOnChats: Delete Row
TagsOnChats-->>Prisma: Confirm Delete
Prisma-->>API: Confirm Delete
API-->>UI: Success Response (204 No Content)
UI->>User: Show Tag Removed from Chat
Explanation: The user interacts with the UI, which calls the NestJS API. The API uses Prisma to either create a new record in the `TagsOnChats` join table (for adding a tag) or delete an existing record (for removing a tag). The `source` could be marked as 'manual'.
sequenceDiagram
participant Scheduler as Scheduler (e.g., EventBridge)
participant HistProcessor as Historical Processor (ECS Task)
participant ChatDataSource as Chat Data Source (DynamoDB Export/SQL)
participant ChunkingSvc as Semantic Chunking Service (TS)
participant EmbeddingSvc as Embedding Service (Python ECS/Lambda)
participant Neo4j as Neo4j Module
participant GDS as Neo4j Graph Data Science Lib
participant GPT4 as GPT-4 API (OpenAI)
participant Prisma as Prisma Client
participant ChatTag as ChatTag Table
Scheduler->>HistProcessor: Trigger Batch Job
loop For each batch of chats
HistProcessor->>ChatDataSource: Read Batch of Historical Chats
ChatDataSource-->>HistProcessor: Return Chat Messages
HistProcessor->>ChunkingSvc: Process Messages into Chunks
ChunkingSvc-->>HistProcessor: Return Chunks
HistProcessor->>EmbeddingSvc: Generate Embeddings
EmbeddingSvc-->>HistProcessor: Return Embeddings
HistProcessor->>Neo4j: Add/Update Chunks & Embeddings
end
Scheduler->>HistProcessor: Trigger Community Detection/Tag Gen Job (less frequent)
HistProcessor->>Neo4j: Call GDS Louvain Algorithm
Neo4j->>GDS: Run Louvain
GDS-->>Neo4j: Return Community Assignments
Neo4j-->>HistProcessor: Confirm Community Detection Ran
HistProcessor->>Neo4j: Identify New Communities without Tags
Neo4j-->>HistProcessor: Return New Community IDs & Sample Chunks
loop For each New Community
HistProcessor->>GPT4: Send Sample Chunks + Prompt for Tags (in primary language)
GPT4-->>HistProcessor: Return Suggested Tag Labels
HistProcessor->>Prisma: Check if Tag Label exists in ChatTag (for group/clinic)
Prisma-->>HistProcessor: Existing Tag or Null
alt Tag Does Not Exist
HistProcessor->>Prisma: Create new ChatTag record
Prisma->>ChatTag: Insert Row
ChatTag-->>Prisma: Return New Tag ID
Prisma-->>HistProcessor: New Tag ID
else Tag Exists
HistProcessor->>HistProcessor: Use Existing Tag ID
end
HistProcessor->>Neo4j: Link Community Node to Tag Node
end
Explanation: A scheduled task processes historical chats in batches, chunking, embedding, and storing them in Neo4j. Periodically, another job runs community detection (Louvain) using Neo4j's GDS library. For new communities found, it samples chunks, asks GPT-4 to generate tag labels, checks/creates the tag definition in the `ChatTag` table via Prisma, and links the community to the tag in Neo4j.