This project is a voice-enabled chatbot that allows users to record their speech, send it to a Flask backend for processing, and receive both text and audio responses. It integrates:
- JavaScript for handling audio recording in the browser π€
- Flask as the backend to process audio and generate responses π₯οΈ
- Speech-to-Text (STT) API for transcribing user speech π
- Text-to-Speech (TTS) API for generating spoken responses π
π project-folder/
βββ π app.py # Flask backend
βββ π templates/
β βββ π index.html # Frontend interface
βββ π static/
β βββ π script.js # JavaScript for recording & sending audio
βββ π requirements.txt # Dependencies
βββ π README.md # Documentation
- The user clicks the Record button.
- JavaScript uses
MediaRecorderto capture microphone input. - The audio is stored as chunks and converted into a
Blob.
- The recorded audio is sent to the Flask backend (
/process_audio) usingFormData. - The Flask app processes the audio using an STT API to extract text.
- The backend analyzes the transcribed text and generates a response.
- A TTS API converts the response into speech.
- The response is displayed as a text message.
- The assistantβs voice response is played using the
<audio>tag in JavaScript.
- Method:
POST - Input: Audio file (OGG/WAV format)
- Output: JSON response containing transcribed text and a base64-encoded audio response
- Example Response:
{
"transcript": "Hello! How can I help you?",
"response_text": "Hi there! I'm your AI assistant.",
"audio_base64": "UklGRgA... (base64 encoded audio)"
}- Uses
navigator.mediaDevices.getUserMedia({ audio: true })to access the microphone. MediaRecordercaptures and encodes the audio.- Uses
fetch()to send the audio to the Flask server. - Dynamically updates the chat UI with messages and plays audio responses.
This project demonstrates how voice-based chat applications can be built using Flask and JavaScript. It's an excellent starting point for more advanced AI-powered conversational assistants. π
π Happy coding! If you have any questions, feel free to ask!