Configuring audio with WebRTC
X2's Speech Recognizer will by default have control of the system microphone. Please make sure the voice recognizer is paused before initiating a WebRTC call as the glasses continuously access the microphone. Such resources (microphone) can’t be shared with multiple processes at the same time. To pause microphone, execute this before initiating/accepting the call:
const val THIRD_EYE_VOICE_COMMAND_INTENT_PACKAGE =
"com.thirdeyegen.api.voicecommand"
fun pauseVoiceRecognizer(context: Context) {
val intent =
Intent(THIRD_EYE_VOICE_COMMAND_INTENT_PACKAGE)
intent.putExtra("instructions", "interrupt")
context.sendBroadcast(intent)
}
And when the call is over, resume the recognizer like this:
fun resumeVoiceRecognizer(context: Context) {
val intent =
Intent(THIRD_EYE_VOICE_COMMAND_INTENT_PACKAGE)
intent.putExtra("instructions", "resume")
context.sendBroadcast(intent)
}
Another issue might be the glasses might be trying to stream audio through the headphone line. To make sure that the glasses are routing audio through the in-built speakers please execute this line during/before the call:
private var audioManager = getSystemService(Context.AUDIO_SERVICE) as
AudioManager
audioManager.isSpeakerphoneOn = true
audioManager.mode = AudioManager.MODE_IN_COMMUNICATION
If the audio has a lot of noise, after making modification specified in the previous sections, please make sure that you are using correct WebRTC audio configuration like this:
AudioDeviceModule adm = createAudioDeviceModule();
peerConnectionFactory = PeerConnectionFactory.builder()
.setVideoEncoderFactory(encoderFactory)
.setVideoDecoderFactory(decoderFactory)
.setAudioDeviceModule(adm)//Important
.setOptions(options)
.createPeerConnectionFactory();
private AudioDeviceModule createAudioDeviceModule() {
return builder(activity)
.setAudioSource(MediaRecorder.AudioSource.MIC)
.setUseHardwareAcousticEchoCanceler(false) //Important
.setUseHardwareNoiseSuppressor(false) //Important
// .setInputSampleRate(16000)
.createAudioDeviceModule();
}
When using Twilio, to make these modifications a developer will have to download the Twilio source code from GitHub and make changes there. This code should be used that code as a dependency, instead of using a pre-built Gradle dependency. This fixes the noise, and is required to be added when creating peer connection factory.