This content originally appeared on DEV Community and was authored by Rugved
In this post, I'll walk you through how to create a voice-based AI career counseling mentor that speaks Hindi using Twilio for voice calls and Groq's LLaMA models for generating AI responses. This bot will act as a friendly and knowledgeable mentor, guiding students through career-related queries in Hindi. We'll also use gTTS for text-to-speech conversion and ngrok to expose our local Flask server to the internet.
Overview of the Tech Stack
- Twilio: Handles voice calls and speech recognition.
- Groq's LLaMA Models: Powers the AI mentor with natural language understanding and response generation.
- gTTS (Google Text-to-Speech): Converts Hindi text into speech.
- Flask: A lightweight Python web framework to handle HTTP requests and responses.
- ngrok: Exposes your local Flask server to the internet so Twilio can interact with it.
How It Works
-
Incoming Call: Twilio forwards the call to your Flask app's
/voiceendpoint. -
Speech Recognition: The app uses Twilio's
Gatherto capture the user's speech. -
AI Response: The app sends the speech text to Groq's LLaMA model, which generates a response in English. The response is then translated to Hindi using
deep-translator. -
Text-to-Speech: The Hindi text is converted to speech using
gTTSand played back to the user. - Continuous Conversation: The bot continues the conversation by prompting the user for more input.
Step-by-Step Implementation
1. Set Up Your Environment
Before diving into the code, ensure you have the following:
- Python 3.10+ installed.
- A Twilio account with a phone number capable of making voice calls.
- A Groq API key (sign up at Groq).
- Install the required Python libraries:
pip install flask twilio groq gtts deep-translator
2. Break Down the Code into Functions
To make the code modular and easier to understand, let's break it into separate functions and explain each part.
a. Initialize Flask and Dependencies
from flask import Flask, request, Response, send_from_directory
from twilio.twiml.voice_response import VoiceResponse, Gather
from twilio.rest import Client
from groq import Groq
import os
from gtts import gTTS
from deep_translator import GoogleTranslator
# Initialize Flask
app = Flask(__name__)
AUDIO_DIR = "static"
os.makedirs(AUDIO_DIR, exist_ok=True)
# Twilio Credentials (Replace with yours)
TWILIO_ACCOUNT_SID = os.getenv('TWILIO_ACCOUNT_SID')
TWILIO_AUTH_TOKEN = os.getenv('TWILIO_AUTH_TOKEN')
TWILIO_PHONE_NUMBER = os.getenv('TWILIO_PHONE_NUMBER')
MY_PHONE_NUMBER = os.getenv('MY_PHONE_NUMBER') # Your personal number
twilio_client = Client(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN)
# Initialize the Groq client
client = Groq(api_key=os.getenv('GROQ_API_KEY')) # Add your API key here
ngrok_url = 'https://44c5-45-117-215-176.ngrok-free.app' # Replace with your ngrok URL
# Messages for Groq conversation context
messages = [
{
"role": "system",
"content": """You are a highly knowledgeable and professional mentor female voice assistant who can solve any query regarding to career choice. Provide insightful, personalized, and practical guidance to students regarding their career and academic path. Focus on delivering clear, concise, expert advice using natural, conversational language suitable for Hindi translation and speech. Be encouraging and supportive. Keep responses short and to the point. Start with the greeting and flow should be human-like. Behave like you are his friend and guide."""
}
]
This section initializes the Flask app, sets up directories, and loads environment variables for Twilio and Groq. It also defines the system prompt for the AI mentor.
b. Handle Incoming Voice Calls
@app.route("/voice", methods=['POST'])
def voice():
"""Handles incoming voice calls from Twilio."""
response = VoiceResponse()
response.say("Namaste, aap kaise hai?", voice="Polly.Aditi", language="hi-IN" )
gather = Gather(input="speech", action="/process_voice", method="POST", language="hi-IN", speechTimeout="auto")
gather.say("कृपया अपनी समस्या बताएं।", voice="Polly.Aditi", language="hi-IN")
response.append(gather)
return Response(str(response), content_type='text/xml')
This function handles incoming voice calls. It greets the user in Hindi and prompts them to describe their career-related query using Twilio's Gather verb for speech input.
c. Process User Speech and Generate AI Response
@app.route("/process_voice", methods=['POST'])
def process_voice():
"""Processes user Hindi speech input, generates a response, and returns a spoken reply."""
response = VoiceResponse()
speech_text = request.form.get("SpeechResult", "")
if not speech_text:
response.say("मुझे आपकी आवाज़ सुनाई नहीं दी, कृपया फिर से बोलें।", voice="Polly.Aditi", language="hi-IN")
response.redirect("/voice") # Retry input
return Response(str(response), content_type='text/xml')
print(f"User said: {speech_text}")
# Generate AI response using Groq
ai_response = generate_response(speech_text)
print(f"AI response: {ai_response}")
# Convert AI response to Hindi speech
speech_file = text_to_speech(ai_response, "hi")
# Stream the response back to the user
response.play(speech_file)
# Continue the conversation
gather = Gather(input="speech", action="/process_voice", method="POST", language="hi-IN", speechTimeout="auto")
response.append(gather)
return Response(str(response), content_type='text/xml')
This function processes the user's speech input, generates an AI response using Groq, converts the response to Hindi speech, and streams it back to the user. If no speech is detected, it prompts the user to try again.
d. Generate AI Response Using Groq
def generate_response(prompt):
"""Uses Groq to generate AI responses in Hindi based on conversation context."""
messages.append({"role": "user", "content": prompt})
try:
ai_response = client.chat.completions.create(
model="llama-3.3-70b-versatile", # Replace with your preferred Groq model
messages=messages,
max_tokens=32768,
temperature=0.5,
n=1
)
res = ai_response.choices[0].message.content.strip()
# Translate to Hindi
translated = GoogleTranslator(source='auto', target='hi').translate(res)
print(f"Translated: {translated}")
messages.append({"role": "assistant", "content": translated})
return translated
except Exception as e:
print(f"Error generating response: {e}")
return "मुझे क्षमा करें, मैं आपकी सहायता नहीं कर सकता।"
This function sends the user's query to Groq's LLaMA model, generates a response, translates it to Hindi, and appends it to the conversation context.
e. Convert Text to Speech
def text_to_speech(text, lang="hi"):
"""Converts text to speech using gTTS and returns the file path."""
speech = gTTS(text=text, lang=lang, slow=False)
speech_file = "static/response.mp3" # Save as MP3 for better compatibility
speech.save(speech_file)
print(f"{ngrok_url}/{speech_file}")
return f"{ngrok_url}/{speech_file}"
This function converts the Hindi text into speech using gTTS and saves it as an MP3 file. It returns the URL of the audio file for Twilio to stream.
f. Make a Test Call
def make_call():
try:
call = twilio_client.calls.create(
url=f'{ngrok_url}/voice', # URL to the TwiML response
to=MY_PHONE_NUMBER, # Replace with the user's phone number
from_=TWILIO_PHONE_NUMBER # Your Twilio phone number
)
print(f"Call SID: {call.sid}")
except Exception as e:
print(f"Error making call: {e}")
This function initiates a test call to your phone number using Twilio's API.
g. Serve Static Files
@app.route('/static/<path:filename>')
def serve_static(filename):
"""Serves static files like audio WAV files."""
return send_from_directory("static", filename)
This function serves static files (e.g., audio files) from the static directory.
h. Run the Application
if __name__ == "__main__":
make_call()
app.run(host='0.0.0.0', port=5000)
This block starts the Flask app and makes a test call to your phone number.
3. Set Up ngrok
Since Twilio needs a publicly accessible URL to interact with your Flask app, we’ll use ngrok to expose your local server.
- Download and install ngrok from ngrok.com.
- Start ngrok with the following command:
ngrok http 5000
- Copy the HTTPS URL provided by ngrok (e.g.,
https://44c5-45-117-215-176.ngrok-free.app) and update thengrok_urlvariable in the Flask app.
4. Configure Twilio
- Log in to your Twilio console.
- Go to Phone Numbers > Manage > Active Numbers and select your Twilio phone number.
- Under the Voice & Fax section, set the Webhook for incoming calls to your ngrok URL (e.g.,
https://44c5-45-117-215-176.ngrok-free.app/voice).
5. Run the Application
- Start your Flask app:
python app.py
- Call your Twilio phone number. The bot will greet you in Hindi and ask for your career-related query.
Challenges and Improvements
- Speech Recognition Accuracy: Twilio's Hindi speech recognition may not always be perfect. You can explore alternatives like Google Speech-to-Text for better accuracy.
- Response Latency: Depending on the Groq model and network, there might be a delay in generating responses. Optimize by using smaller models or caching frequent queries.
- Dynamic ngrok URLs: If you restart ngrok, the URL changes. Consider using a paid ngrok plan for static URLs.
Conclusion
By combining Twilio, Groq's LLaMA models, and gTTS, you can create a powerful Hindi-speaking AI career counseling mentor. This project demonstrates the potential of AI in providing personalized guidance in regional languages, making it accessible to a wider audience. Feel free to extend this project by adding more features like multi-language support or integrating with other APIs.
Let me know in the comments if you have any questions or ideas for improvement!
GitHub Profile
LinkedIn Profile
Happy coding! 👩💻👨💻
This content originally appeared on DEV Community and was authored by Rugved
Rugved | Sciencx (2025-02-10T17:53:32+00:00) Building a Hindi-Speaking AI Career Counseling Mentor with Twilio and Groq’s LLaMA Models. Retrieved from https://www.scien.cx/2025/02/10/building-a-hindi-speaking-ai-career-counseling-mentor-with-twilio-and-groqs-llama-models-2/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.