Skip to content

Commit ac4d22a

Browse files
authored
Merge pull request #92 from Project-Babble/main
Baballonia release
2 parents d8e6ff5 + f16aab4 commit ac4d22a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+35540
-1250
lines changed

docs/babbleofficaltracker/babbleofficaltracker.mdx

Lines changed: 83 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -7,46 +7,86 @@ title: Babble Tracker Quickstart
77

88
This guide applies to the Official Babble Tracker, and should cover DIY versions that use similar hardware/firmware. It will walk you through the complete setup process, from initial connection to calibration and troubleshooting.
99

10+
11+
:::info
12+
If you'd like to use your tracker in *Wireless* mode, follow this [guide.](docs/babbleofficaltracker/babbleofficaltrackerwireless.mdx)
13+
:::
1014
## Getting Started
11-
### Initial Setup
15+
### Hardware Setup
1216

13-
1. **Plug your tracker into your PC**
14-
- Connect using the USB port on your tracker
15-
- Make sure the switch is in the "ON" position, as seen below
16-
- Two bright green lights should appear on the back of the tracker
17-
- You should hear a connection sound when properly connected
17+
**Plug your tracker into your PC**
18+
1) Connect using the USB port on your tracker
19+
1) Make sure the switch is in the "ON" position, as seen below
20+
1) A bright green lights should appear on the back of the tracker
21+
1) You should hear a connection sound when properly connected
22+
1) The tracker will take about 20 seconds to boot upon receiving power
1823

1924
![Switch Position](/img/setup_switch.jpg)
2025

21-
2. **Finding your device's name**
22-
- **Windows**
23-
- Press the Windows key, type "Device Manager" and press enter
24-
- Expand "Ports (COM & LPT)"
25-
- Look for a "USB Serial Device" entry - this is your tracker's COM port
26+
### Software Setup
27+
#### Windows
28+
- Get the latest version from our [Github](https://github.com/Project-Babble/Baballonia/releases/latest). Download the installer and run it.
29+
#### macOS/Linux
30+
- Currently, we do not offer installers for these platforms. You will need to build these from source.
31+
- For assistance building Babble Avalonia on these platforms, please contact us in our discord server.
32+
- Our [Legacy App](docs\software\babbleApps\babble-app.mdx) can be run from our [installer scripts](https://github.com/project-Babble/projectBabble/?tab=readme-ov-file#macos-and-linux) on the Github
33+
34+
35+
### Mode Selection
36+
37+
:::info
38+
For more detailed instructions and troubleshooting information, head to the [firmware guide.](/hardware/Firmware/firmware.mdx)
39+
:::
40+
41+
42+
Now that you've got your Babble Tracker, you'll need to flash the firmware for it.
43+
44+
#### Step 1 - Open Baballonia and navigate to the Firmware page:
45+
46+
#### Step 1.1 - Plug in *or* unplug and replug your tracker to make sure it's in configuration mode.
47+
48+
![fw](@site\static\img\avalonia\firmware\new\fw.png)
49+
50+
#### Step 2 - Refresh the connected trackers. Then, select yours:
51+
52+
:::warning
53+
Upon connecting the tracker, you will have a 20 second window to press ‘Refresh’. If you wait too long, you will need to unplug the tracker and try again.
54+
:::
55+
56+
![refresh](@site\static\img\avalonia\firmware\new\refresh.png)
57+
![dropdown](@site\static\img\avalonia\firmware\new\dropdown.png)
58+
![selected](@site\static\img\avalonia\firmware\new\selected.png)
59+
60+
#### Step 3 - Set your tracker into 'Wired' mode:
61+
62+
![mode](@site\static\img\avalonia\firmware\new\mode.png)
63+
2664

27-
![Device Manager](/img/setup_comm.png)
65+
![wifi](@site\static\img\avalonia\firmware\new\wifi.png)
2866

29-
- **macOS/Linux**
30-
- On **macOS/Linux**, the Babble Tracker with appear as a device with a `/dev/ttyACM` prefix . You can list what devices are present with `ls -ld /dev/ttyACM*`.
67+
#### Step 4 Unplug and replug your tracker. 20 seconds after being powered on, it will start!
3168

32-
#### Download and install the software
33-
- **Windows**
34-
- Get the latest version from here [Babble Tracker Software](https://drive.google.com/file/d/1KmMBYSmAGlqDeTEjgnOOCX8YpWeeBgPb/view?usp=drive_link). Download the Project_Babble-Setup-vX.X.X.exe file and run the installer
35-
- **macOS/Linux**
36-
- Use the [installer scripts](https://github.com/project-Babble/projectBabble/?tab=readme-ov-file#macos-and-linux) on the Github
69+
:::info
70+
The tracker will now boot into UVC mode and function as a USB camera with the device name 'openiristracker'
71+
:::
3772

38-
#### Connect to your tracker
39-
- Open the Project Babble software
40-
- Click on the dropdown next to "Camera Address"
41-
- Select the namely identified in step 2
42-
- Click "Save and Restart Tracking" to connect
4373

74+
### Connect to your tracker
75+
- Open Baballonia
76+
- Open the dropdown menu
77+
- Select 'openiristracker'
4478
:::info
45-
If your camera isn't present in the dropdown, just type it in.
79+
Older trackers and DIY solutions running serial firmware will be start with `COM`/`/dev/ttyACM` followed by a number
4680
:::
81+
- Click "Start Camera" to connect
4782

48-
![Camera Address Selection](/img/setup_dropdown_comm_select.png)
83+
![Camera Address Selection](/img/uvcfirst.png)
4984

85+
![Camera Address Selection](/img/uvcsecond.png)
86+
87+
![Camera Address Selection](/img/uvcthird.png)
88+
89+
![Camera Address Selection](/img/uvclast.png)
5090
## Setting Up The Camera Feed
5191

5292
Before use, you need to properly set up your camera feed:
@@ -57,124 +97,44 @@ Before use, you need to properly set up your camera feed:
5797
- Positioned at a slight angle that shows the tip of your nose/nostrils
5898

5999
2. **Set the crop area**
60-
- Click the "Cropping Mode" button
100+
- Set the Face Camera into "Cropping" mode
61101

62-
![Cropping Mode Button](/img/babbleofficaltracker/1.png)
102+
![Cropping Mode Button](/img/crop1.png)
63103

64-
- The feed may appear choppy in this mode - *this is normal*
65104
- Click and drag across the image to set the crop area
105+
106+
107+
![Cropping Mode Button](/img/crop2.png)
108+
66109
- Make sure the cropped area includes:
67110
- Tip of your nose/nostrils
68111
- Bottom of your jaw when open
69112
- Sides of your cheeks when puffed
70113

71114
![Proper Crop Example](/img/babbleofficaltracker/2.png)
72115

73-
- Don't worry if the image is flipped - mirroring options are available at the bottom of the cam tab
116+
- Don't worry if the image is flipped - mirroring options are available in the "Face Camera Settings" dropdown
74117

75-
3. **Switch to tracking mode**
76-
- Once you've set your crop, click "Tracking Mode"
77-
78-
![Tracking Mode Button](/img/babbleofficaltracker/3.png)
79-
80-
- You should now see a smooth video feed from your camera
81-
82-
![Working Camera Feed](/img/setup_working_V.png)
118+
![Face Camera Settings](/img/facecam.png)
83119

84-
## Calibration Process
120+
![Face Camera Settings](/img/avaloniaface.png)
85121

86-
Proper calibration is essential for accurate facial tracking:
122+
3. **Switch to tracking mode**
123+
- Once you've set your crop, set your face camera back into "Tracking"
87124

88-
1. **Navigate to the Calibration tab**
89125

90-
![Calibration Tab](/img/babbleofficaltracker/4.png)
91-
92-
2. **Perform Neutral Calibration**
93-
- Set the calibration mode to "Neutral"
94-
95-
![Neutral Mode](/img/babbleofficaltracker/5.png)
96-
97-
- Return to the Cam tab
98-
- Check "Enable Calibration"
99-
100-
![Enable Calibration](/img/babbleofficaltracker/8.png)
101-
102-
- Relax your face into its natural resting position
103-
- Click "Start Calibration"
104-
105-
![Start Calibration](/img/babbleofficaltracker/9.png)
106-
107-
- You'll hear a sound indicating calibration has started
108-
- After a few seconds, click "Stop Calibration"
109-
110-
![Stop Calibration](/img/babbleofficaltracker/10.png)
111-
112-
3. **Perform Full Calibration**
113-
- Return to the Calibration tab
114-
- Set the calibration mode to "Full"
115-
116-
![Full Mode](/img/babbleofficaltracker/6.png)
117-
118-
- Go back to the Cam tab
119-
- Click "Start Calibration"
120-
- After hearing the start sound, make as many facial expressions as possible:
121-
- Smile, frown, open mouth wide
122-
- Puff cheeks, stick out tongue
123-
- Move jaw side to side
124-
- **Tip:** Keep movements natural - don't exaggerate expressions
125-
- Calibration ends when you hear a different sound
126-
127-
## Fine-Tuning and Troubleshooting
128-
129-
If some expressions work better on one side than the other, or you need to adjust tracking sensitivity:
130-
131-
### Understanding the Calibration Tab
132-
133-
![Calibration Panel](/img/babbleofficaltracker/12.png)
134-
135-
The calibration panel allows you to manually adjust tracking values:
136-
137-
- **Left/Right**: Indicates which side of the face the shape corresponds to
138-
139-
![Left Right Options](/img/babbleofficaltracker/13.png)
140-
141-
- **Shapes**: The specific facial movement (left and/or right) being tracked (Mouth Open, Cheek Puff, etc.)
142-
143-
![Shape Names](/img/babbleofficaltracker/14.png)
144-
145-
- **Min**: The threshold at which the model starts detecting a shape
146-
147-
![Min Value](/img/babbleofficaltracker/15.png)
148-
149-
- **Max**: The maximum value the model detects for the shape
150-
151-
![Maximum Value](/img/babbleofficaltracker/16.png)
152-
153-
### Adjusting Values
154-
155-
- For proper tracking, minimum values should generally start at around 0.1
156-
- Example with CheekPuff:
157-
- Setting Minimum to 0.4: You need significant cheek puffing before detection
158-
- Setting Maximum to 0.1: Slight cheek movement registers as full puffing
159-
160-
### Fixing Common Issues
161-
162-
If your jaw appears to lean in your neutral position:
163-
- Adjust the minimum value for JawLeft until it's no longer detected when your face is at rest
164-
165-
**Pro Tip:** For precise adjustments, open your VR application's debug menu for the mouth tracking asset. Watch how values change as you make expressions, then return to the calibration tab to fine-tune.
166-
167-
![Maximum Value](/img/babbleofficaltracker/17.png)
126+
- You should now see a smooth and cropped video feed from your camera
127+
128+
![Working Camera Feed](/img/crop3.png)
168129

169130
## Next Steps
170131

171-
Your Babble Tracker is now set up and ready to use! To get started with specific VR platforms:
132+
To learn more about the Babble App, [click here.](/docs/software/baballonia.mdx)
133+
134+
To get started with specific VR platforms:
172135

173136
- [VRChat Integration Guide](docs/software/integrations/vrc.mdx)
174137
- [ChilloutVR Integration Guide](docs/software/integrations/chilloutVR.mdx)
175138
- [Resonite Integration Guide](docs/software/integrations/resonite.mdx)
176139
- [Other Platforms](docs/software/integrations/integrations.mdx)
177140

178-
## WiFi Capability
179-
180-
Want to use your tracker wirelessly? Check out our [Firmware Guide](docs/hardware/Firmware/firmware.mdx) for instructions on enabling and configuring WiFi functionality.

0 commit comments

Comments
 (0)