AI in VR, Building a Speech-Driven NPC with GPT/LLM: A Step-by-Step Tutorial

Join us on this in-depth tutorial as we explore the world of speech-driven NPCs and learn how to create our own using GPT/LLM. From building the architecture and design to implementing speech synthesis and natural language processing, we'll cover every step of the process.

Get hands-on experience with top APIs like Hugging Face, GroqCloud, and Speechify, and discover how to bring your characters to life with lip-sync and animation.

In this video, you'll learn:

How to build a humanoid NPC with AI-generated speech
How to setup and configure lip-sync and animation for realistic character interactions
How to integrate speech synthesis and natural language processing
The top APIs and tools for building speech-driven NPCs

Whether you're a game developer, animator, or AI enthusiast, this tutorial is designed to help you unlock the power of speech-driven NPCs and take your projects to the next level.

So, what are you waiting for? Let's get started and explore the world of speech-driven NPCs today!
====================================

Link to a great uLipSync tutorial: https://youtu.be/k5CtTsIKwE4?si=NWwznsQCG4PscyUP
Link to SmartNPC branch on GitHub: https://github.com/digitalplusplus/BaM-Construct/tree/SmartNPC
Join discord: https://discord.gg/A85YG5ch

00:00 Intro
01:55 BaM Teaser & Music
02:34 Special Thanks
02:54 Research
11:43 Architecture & Design
18:00 Build/Coding Review
20:11 HuggingFace API
22:56 GroqCloud API
23:41 Speechify API
24:38 TTS REST API & Audio Output Component
32:56 TTS Testing
33:22 LLM REST API & JSON Component
39:45 LLM Testing
40:22 LLM+TTS Testing
40:44 STT REST API & Microphone Component
44:59 STT Testing
46:13 Building a Humanoid NPC
50:24 Adding LipSync
52:19 Adding Animation
54:08 The NPC Lost It!
54:53 Animation Control by TTS
55:53 First E2E Test
56:14 Syncing BlendShapes
58:18 Second E2E Test
58:33 Finetune LipSync
58:50 Final E2E Test!
1:00:24 Final words
=======================================
AI, GenerativeAI, VR, TTS, STT, LLM, Unity, vr, vr tutorials, virtual reality, Unity tutorials, xr, vr tutorial, ai tutorial, vr tutorial for beginners, vr tutorial unity, unity, unity vr, unity vr tutorial, beginner, basic, learn, vr programming, how, to, howto, gamedev, vr dev, vr development, indie gamedev, beginner xr tutorial, unity xr, xr toolkit unity, REST API, REST, rest, rest api, Groq, Speechify, Llama, llama, LLaMA, Llama 3.1, Meta, Pico, Quest, quest, HuggingFace, huggingface, rapidapi, RapidAPI, OpenAI, Speech to Text, Text to Speech, TTS, STT, uLipSync, ulipsync, lipsync, blendshapes

Тэги:

##AI ##LLM ##Unity ##VR ##STT ##speechify

Скачать видео