Caution
This version of the software is deprecated and no longer maintained. Please visit the new repository for the latest version of the GPT Assisted, Lip Synced Chatbot project.
We strongly recommend using the updated version to ensure you have access to the latest features and improvements.
LipSync ChatBot is an innovative project that integrates Python's speech recognition capabilities with a visualized character able to lip-sync in Unity. The chatbot creates a seamless interaction experience, with the lips of the character syncing perfectly with the audio. The future development plan includes the improvement to work solely with Unity using the ChatGPT API.
LipSync ChatBot offers a realistic and engaging user experience, making it suitable for games, simulations, and more.
- Speech recognition using Python
- Lip synchronization with visualized characters in Unity
- Flexibility in adjusting lip sync power and distance
- Compatibility with wav file formats
- Open the Unity project: Prepare the Unity environment.
- Run the Python project: Activate the speech recognition module.
- Unity Detection over localhost: Unity will detect the Python project running over localhost.
- TTS and AudioSegment Library: Python uses TTS to create sound, then the AudioSegment library converts the mp3 file to a wav file.
- Unity Reads Sound File: Python sends a command to Unity, letting it read the sound file from the filesystem and make the model lip-sync to the sound.
* volume: Change the power of lip sync (distance between lips, etc.).limit = Mathf.clamp: Adjust the line's end to change the maximum distance the lips can get.
- Integration with ChatGPT API to make the chatbot much more intelligent and can be only run on Unity without Python project.
Feel free to contact me if you have any questions or need further assistance with this project.