STT service fails when screen is turned off or locked

[dp0s]

Hey there, first of all thanks for the continuous work on this app. I'm very happy to see that a speech service for other apps was included lately.

I started to integrate Dicio's speech recognition into Tasker using the "Get Voice" task. It works reliable and quick enough on my device (Lineage OS 19).

The only problem occurs when I activate it while the screen is turned off, or turned on but locked. Then it aborts listening or fails to hear anything.

Here's an example task to reproduce the issue:

    Task: GetVoice_test
    
    A1: Wait [
         MS: 0
         Seconds: 3
         Minutes: 0
         Hours: 0
         Days: 0 ]
    
    A2: Beep [
         Frequency: 1200
         Duration: 500
         Amplitude: 54
         Stream: 3 ]
    
    A3: Get Voice [
         Language Model: Free Form
         Maximum Results: 1
         Timeout (Seconds): 10
         Continue Task After Error:On ]
    
    A4: Beep [
         Frequency: 1200
         Duration: 500
         Amplitude: 54
         Stream: 3 ]
    
    A5: Say [
         Text: %gv_heard
         Engine:Voice: default:default
         Stream: 3
         Pitch: 5
         Speed: 5
         Respect Audio Focus: On ]
    

When I start this task with screen on, it works fine. when I turn off or lock the screen before the Get Voice command is finished, it fails to hear anything.

To narrow down the problem I tried 2 things:

• Set Google speech services as backend for Tasker's Get Voice command. In this case the above task works even if the screen is off or locked. (the big downside: google requires internet connection for this, so they can collect everything I say. This is annoyingly pointless as the google voice input keyboard works completely offline after downloading the language model.)
• replacing the Get Voice command by a Launch Dicio App command. the app is indeed able to recognize my speech with screen off/locked.

I have granted the "Use Microphone only while using the app" permission to both Dicio and Google Speech Services.

[dp0s]

Btw, this is not Tasker specific. STT services fails too if I press the microphone button in my contacts app and quickly turn off the screen. But Tasker is the actual use case where I need speech recognition with screen off.

[nebkrid]

Hi,
I tried to reproduce it on my phone - not with Tasker but with a similar automation app. But I couldn't get a speech input from google neither (Neither locking screen while speech input is running nor starting a delayed request while screen is on but locked.)
Because there are multiple ways how such a speech input request is generated (StartActivityForResult, RecognizerIntent, background speech service) and I do not know which one Taskers GetVoice uses: In the google case: Does the google voice input screen show up or is it "invisible" in background when you are using it without locking?

The technical part of the issue seems to be that if the screen is turned off the onStop() method of the Dicio Voice Input Activity is called which cancels the speech input. I am not sure how google avoids this since this is usually the wanted behavior if the activity is not stopped due to screen turning off but because the user touched somewhere outside the speech input dialog.

[dp0s]

Hey nebkrid, thanks for looking at this.

When Google STT is used as backend for Tasker with screen on, there is a pop up in the middle of the screen (microphone symbol). As mentioned, it also works when screen is locked/off.

I tried with another app "Automate", where its called the Speech Recognition block. This one uses Google without asking, so I can't select Dicio. There is no visible pop up at all, but it works (also with screen locked/off). As usual, internet connection is required.

For comparison, the "Get Voice" task from Tasker opens a system dialog at first use (choose which app to perform this action with, Just Once/Always). The same happens when I click on the microphone button in the contacts app. The choice is saved system wide.

[nebkrid]

there is a pop up in the middle of the screen

Ok, then it is generally the same way of android request like the one dicio has implemented currently.

As mentioned, it also works when screen is locked/off.

This is the issue what I can not test, because on my device the google one immediatly finishes too, when it is started by a delayed task when the screen is off (I can here the start sound but immediatley the end sound.) Therefore I am currently not very optimistic that I can fix this to behave identically on you phone... Generall

I tried with another app "Automate", where its called the Speech Recognition block. This one uses Google without asking, so I can't select Dicio. There is no visible pop up at all, but it works (also with screen locked/off). As usual, internet connection is required.

This is indeed the automation app I am using, too. I observed the same behaviour, though I am not sure why it is this way. Maybe automate explicitly calls the google service. However, I could overcome this by using an "App decision?" block with Action="android.speech.action.RECOGNIZE_SPEECH" and extracting the spoken text from the extras dicitionary with extras["android.speech.extra.RESULTS"]. This way it works if the screen is on (but not off - so effectivley the same like the Tasker GetVoice...)

=> So, unfortunatley, I have no idea how to get it solved with the popup since I can not compare / test it. But for future it might be possible with the way of background request which is in development. See #161 and the discussion in #151. You can even give the test app a try, but this is on an very early stage.