⚖️ Indian LAW AI-Agent

This AI ChatBot is designed to be private and can answer your queries on Indian Constitution & Indian Penal Code (IPC). This app is using Google ADK for the frontend & LiteLlm wrapper which is build with Ollama LLMs which are namely llama3.2 & mxbai-embed-large .

📽️ Demo

You can play below Video or click this Youtube Link to see the demo. Please let me know in the comments, how do you feel about this App.

🖧 Our Scematic Architecture

This is the scematic flow diagram
Flow-Diagram

⚠️ Disclaimer

This AI application is creating its own RAG database based on two markdown files: Indian Constitution & Bharatiya Nyaya Sanhita (IPC) locaded here . This markdown files were generated using the docling project from the PDF files which are publicly available. The author is not responsible for any wrong information given by this AI Agent.

📦 Quickstart Guide

You can run it on Google Colab and you can run it on a Free GPU machine with the available code . If you want to deploy this complete application on the Container World like Docker, Kubernetes etc. you may follow the next steps.

📘 Code Link

Here is the full codebase on GitHub

git clone https://github.com/daslearning-org/ai-indian-law.git

🚀 Run it on Google Colab

You can access Google Colab for free if you have a Google/Gmail account.
You can download the code & run it or you can directly open it with below option.

☸️ Run it on Kubernetes (example is on GKE)

The example is on GKE, but you can replicate it on any K8S environment. You need to change (or remove) some parameters tolerations , gke-accelerator , gke-spot etc. in the YML files in k8s directory.

Get the kubectl setup for your GKE cluster (only for GKE)

# get the cluster context (only for GKE)
gcloud container clusters get-credentials ai-gke --region=asia-south1 --project=dl-k8s-dev1cade # update with your gke & project details

Run the Ollama backend
```
kubectl apply -f ./k8s/ollama-app.yml
```
Run litellm app
```
kubectl apply -f ./k8s/litellm-app.yml
```

Deploy the UI on K8S

# create namespace for adk (need to add the secret before we can deploy the yml)
kubectl create namespace "adk"

# create k8s secret for litellm api key
kubectl create secret generic litellm-api-key --from-literal=daslearning="YOUR_API_KEY" -n adk

# deploy the UI
kubectl apply -f ./k8s/adk-app.yml

Then get your Load balancer IP address & access it from your browser. Optionally you can map it with your domain.

🐋 Run using docker

Run the Ollama backend on a container and make sure nvidia-container-toolkit is installed if you have Nvidia GPU. The below image will automatically download the required models during startup which may take some time to be ready, depending on internet speed.

docker pull sdas92/ollama-llama3.2-mxbai-embed:v1 # check for the latest version at docker hub
# for GPU machines
docker run -d --gpus=all -p 11434:11434 --name ollama sdas92/ollama-llama3.2-mxbai-embed:v1
# for CPU only machines
docker run -d -p 11434:11434 --name ollama sdas92/ollama-llama3.2-mxbai-embed:v1

Run the LiteLlm backend (which can also be accessed via API calls). In the Ollama URL, use the container IP, do not use localhost as it will try access the ollama within litellm container

docker pull sdas92/law-litellm:v1
docker run -d -p 4000:4000 -e OLLAMA_API_BASE="http://your-ollama-host:11434" --name litellm-rag sdas92/law-litellm:v1 # use the container IP

# You can test or use the API direcly with your application, sample format
curl -X POST http://localhost:4000/chat/completions -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' -d '{"model": "indian-law-llm", "messages": [{"role": "user", "content": "Punishment for money fraud?"}]}'

Run the UI & again use the contair IP litellm-rag in the OpenAI host URL insread of localhost

docker pull sdas92/law-ai-adk:v2

docker run -d -p 8000:8000 -e OPENAI_BASE_URL="http://your-litellm-host:4000" -e OPENAI_API_KEY="sk-1234" --name law-ui sdas92/law-ai-adk:v2 # use litellm container IP in the URL

Open your browser & type http://localhost:8000 to chat with the AI Model.

⚒️ Manual execution on local system or Development / Update

Prerequisites

Run Ollama Backend

Install Ollama on your system from the official website

Then Pull the required models

ollama pull llama3.2
ollama pull mxbai-embed-large
# Check the models
ollama list

Backend Service

Use the backend service in separate terminal than the frontend

Set ENV vars

export OLLAMA_API_BASE="http://localhost:11434"

Run the backend service in separate terminal

cd ./backend/markDownRAG/
python -m venv .venv
source .venv/bin/activate # use .\venv\Scripts\activate on windows
pip install -r requirements.txt
# run the litellm service
litellm --config config.yaml

Frontend Service

Run frontend in another terminal

Set ENV vars

export OPENAI_API_KEY="your-key" # create your key from litellm or use  default 'sk-1234'
export OPENAI_BASE_URL="http://localhost:4000"

Run the frontend service

cd ./frontned
python -m venv .venv
source .venv/bin/activate # use .\venv\Scripts\activate on windows
pip install -r requirements.txt
# run the uvicorn service for ADK
sh -c "uvicorn main:app --host 0.0.0.0 --port 8000" # access the UI at localhost:8000

Access the UI

Open browser & type http://localhost:8000

🐋 Building the docker images & push

You may create your own images & store in your repo

Ollama backend

cd ./backend/ollama/
export APP_VERSION="v1"
export IMAGE_URI="sdas92/ollama-llama3.2-mxbai-embed:${APP_VERSION}" # change to your repo URI
docker build -t ${IMAGE_URI} .
docker push "${IMAGE_URI}"

LiteLlm backend

cd ./backend/markDownRAG/
export APP_VERSION="v1"
export IMAGE_URI="sdas92/law-litellm:${APP_VERSION}" # change to your repo URI
docker build -t ${IMAGE_URI} .
docker push "${IMAGE_URI}"

Frontend UI

cd ./frontend/
export APP_VERSION="v1"
export IMAGE_URI="sdas92/law-ai-adk:${APP_VERSION}" # change to your repo name
docker build -t ${IMAGE_URI} .
docker push "${IMAGE_URI}"