ZenGPT API Server

Getting Started

Set up environment variables:

ENV=Dev  # Staging/Production
OPEN_AI_KEY=
DEFAULT_TOKEN_QUOTA=500000
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=us-east-1
PRE_SHARED_KEY=
MONITOR_ENABLE=True
MONITOR_AXIOM_DATASET=gptdock
MONITOR_AXIOM_API_TOKEN=
MONITOR_AXIOM_ORG_ID=zenumltest-kes2
MONITOR_HEARBEAT_URL=https://ai-staging.gptdock.com
MONITOR_HEARBEAT_INTERVAL_SECONDS=60
MONITOR_DEFAULT_MISFIRE_GRACE_TIME_SECONDS=60
MONITOR_USERS_COUNT_INTERVAL_SECONDS=3600
MONITOR_PLATFORM=API-Server # ACE-Server

The server is using DynamoDB as the database, so you need to set up the AWS credentials. Also you need to set up the OpenAI API key and the pre-shared key for the API.

Both BFF and API server are using the same pre-shared key. Any request without the pre-shared key will be rejected as 403.

Create a virtual environment and install the dependencies:

python -m venv venv
source venv/bin/activate  #windows： venv\Scripts\activate
pip install -r requirements.txt

Start Server:

python src/app.py

Deployment

Railway

AWS Lightsail

server deployed on AWS LightSail us-east-1

domain: no domain yet, this service runs on port 5001 and is not exposed to the internet.

IP address:

3.215.107.112
2600:1f18:115a:4d00:44bb:def9:38a7:306b

API reverse proxy: Caddy [not used]
Language: Python 3.8
SSL: Caddy Let's encrypt
API Service: Gunicorn (with gevent)
API Framework: Flask
Service manager: Supervisor

This service is a stateless service, not necessary for auto snapshot and backup on server.

Deployment steps

Create a Lightsail instance
SSH into the instance
Clone this repo
Install dependencies
Install Caddy
Install Supervisor
Install Gunicorn
Install Gevent
Install Python 3.8
Follow the instructions in the Getting Started section to set up the environment variables
Copy the gptdock-supervisor.conf file to /etc/supervisor/conf.d/
Update supervisor config: sudo supervisorctl reread && sudo supervisorctl update

API

`GET /v1/health`

curl -X GET http://localhost:5001/v1/health

Response:

{
  "status": "ok"
}

`GET /v1/client/info`

curl --request GET \
  --url "http://localhost:5001/v1/client/info?client_id=<client_id>&product_id=<product_id>" \
  --header 'Authorization: Bearer <your_pre_shared_key>'

client_id: the client id of the user
product_id: the product id of the user

Response example:

{"client_id": "1234", "quota_used": 14061, "max_quota": 500000}

or 404 if the client id is not found

`POST /v1/ask`

curl --request POST \
  --url http://localhost:5001/v1/ask \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <your_pre_shared_key>' \
  --data '{
  "question": "Who are you?",
  "user_id": 1,
  "client_id": 1,
  "product_id": 1,
  "stream": false
}'

The request body is a JSON object with the following fields:

question: the question to ask the model
history (optional): a list of objects to including the history of the conversation
product_id: the product id of the user
client_id: the client id of the user
user_id: the user id of the user
stream: whether to stream the response or not

Body example:

{
  "question": "Who are you?",
  "user_id": 1,
  "client_id": 1,
  "product_id": 1,
  "stream": false,
  "history": [
    {
      "role": "user",
      "content": "hello."
    },
    {
      "role": "assistant",
      "content": "Hi."
    }
  ]
}

Response example:

{
  "id": "chatcmpl-709BzPEcjA6tscXpFKX6m8E3WKCxa",
  "object": "chat.completion",
  "created": 1680269691,
  "model": "gpt-3.5-turbo-0301",
  "usage": {
    "prompt_tokens": 118,
    "completion_tokens": 57,
    "total_tokens": 175
  },
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "I am GPTDock, an AI assistant powered by the language model gpt-3.5-turbo. I am designed to assist you in a wide range of tasks, from answering questions to generating creative ideas and assisting with problem-solving. How may I assist you today?"
      },
      "finish_reason": "stop",
      "index": 0
    }
  ]
}

Response as Stream

curl --request POST \
  --url http://localhost:5001/v1/ask \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <your_pre_shared_key>' \
  --data '{
  "question": "Who are you?",
  "user_id": 1,
  "client_id": 1,
  "product_id": 1,
  "stream": true
}'

Response example:

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"role": "assistant"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "Hello"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "!"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " I"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " am"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " G"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "PT"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "Dock"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": ","}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " an"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " AI"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " assistant"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " based"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " on"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " the"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " language"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " model"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " g"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "pt"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "-"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "3"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "."}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "5"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "-t"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "ur"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "bo"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "."}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " I"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " am"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " here"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " to"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " help"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " you"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " with"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " any"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " questions"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " or"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " tasks"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " you"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " may"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " have"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "."}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " Just"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " let"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " me"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " know"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " how"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " I"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " can"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " assist"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": " you"}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {"content": "."}, "index": 0, "finish_reason": null}]}

data: {"id": "chatcmpl-709fWs9gLwfczWRasDCadymJm3vEG", "object": "chat.completion.chunk", "created": 1680271522, "model": "gpt-3.5-turbo-0301", "choices": [{"delta": {}, "index": 0, "finish_reason": "stop"}]}

data: [DONE]

Note: The stream respond will be multiple SSE events, each event is an utf8 string starting with data: . and ending with \n\n. The data is a JSON object with the following fields:

{
  "choices": [
    {
      "delta": {
        "content": "c"
      },
      "finish_reason": null,
      "index": 0
    }
  ]
}

Name		Name	Last commit message	Last commit date
Latest commit History 85 Commits
.github		.github
infra		infra
scripts		scripts
src		src
.gitignore		.gitignore
Caddyfile		Caddyfile
README.md		README.md
gptdock-supervisor.conf		gptdock-supervisor.conf
nixpacks.toml		nixpacks.toml
requirements.txt		requirements.txt
sample.env		sample.env

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

ZenGPT API Server

Getting Started

Deployment

Railway

AWS Lightsail

Deployment steps

API

`GET /v1/health`

`GET /v1/client/info`

`POST /v1/ask`

Response as Stream

About

Uh oh!

Releases 2

Packages

Uh oh!

Contributors

Uh oh!

Languages

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

ZenGPT API Server

Getting Started

Deployment

Railway

AWS Lightsail

Deployment steps

API

GET /v1/health

GET /v1/client/info

POST /v1/ask

Response as Stream

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases 2

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`GET /v1/health`

`GET /v1/client/info`

`POST /v1/ask`

Packages