Flutter:Custom Voice Bot in Background like Google assistant or SIRI.

#flutter #dart #programming #tutorial

In this tutorial we are going to create an application for voice bot that will listen voice in background or foreground and will answer to submitted query.
I will be focusing on Android only because i never test application in iOS(Not enough money to have).

Lets Start Coding-

1) First of all create a new flutter application like this

2) Add following package in your pubspec.yaml file:



- flutter_tts: ^3.3.3 (use for text to speech)

- flutter_background_service: ^2.1.0 (use for handling app in background)

- speech_to_text: (use for speech to text)

3). Android configuration:

`Change the minimum Android SDK version to 21 (or higher) in your android/app/build.gradle file.

minSdkVersion 21

Note:Apps targeting Android 11 that use text-to-speech should declare TextToSpeech.Engine.INTENT_ACTION_TTS_SERVICE in the queries elements of their manifest.`



<queries>
<intent>
<action android:name="android.speech.RecognitionService"/>
</intent>
</queries>

Add following in android/app/src/main/AndroidManifest.xml



<uses-permission android:name="android.permission.RECORD_AUDIO"/>
    <uses-permission android:name="android.permission.INTERNET"/>
    <uses-permission android:name="android.permission.BLUETOOTH"/>
    <uses-permission android:name="android.permission.BLUETOOTH_ADMIN"/>
    <uses-permission android:name="android.permission.BLUETOOTH_CONNECT"/>

4)Now run command in terminal
flutter pub get

5)Now create a new file called 'background_service.dart' in same directory.

6)We will be first setup Background handling so write following code in new file and most important do forgot to import all the require files.



final service = FlutterBackgroundService();

Future initializeService()async{
  await service.configure(
    androidConfiguration: AndroidConfiguration(
      // this will executed when app is in foreground or background in separated isolate
      onStart: onStart,
      // auto start service
      autoStart: true,
      isForegroundMode: true,
    ),
    iosConfiguration: IosConfiguration(
      // auto start service
      autoStart: true,

      // this will executed when app is in foreground in separated isolate
      onForeground: onStart,

      // you have to enable background fetch capability on xcode project
      onBackground: onIosBackground,
    ),
  );
  await service.startService();
}
bool onIosBackground(ServiceInstance service) {
  WidgetsFlutterBinding.ensureInitialized();
  print('FLUTTER BACKGROUND FETCH');

  return true;
}
void onStart(ServiceInstance service) async {

  // Only available for flutter 3.0.0 and later
  DartPluginRegistrant.ensureInitialized();

  // For flutter prior to version 3.0.0
  // We have to register the plugin manually

  if (service is AndroidServiceInstance) {
    service.on('setAsForeground').listen((event) {
      //set as foreground
      service.setAsForegroundService();
    });

    service.on('setAsBackground').listen((event) async {
      //set as background
      service.setAsBackgroundService();
    });
  }

  service.on('stopService').listen((event) {
    service.stopSelf();
  });
  // bring to foreground
  Timer.periodic(const Duration(seconds:1), (timer) async {
    if (service is AndroidServiceInstance) {
      service.setForegroundNotificationInfo(
        title: "My App Service",
        content: "Updated at ${DateTime.now()}",
      );
    }

    /// you can see this log in logcat
    print('FLUTTER BACKGROUND SERVICE: ${DateTime.now()}');

    // test using external plugin
    service.invoke(
      'update',
      {
        "current_date": DateTime.now().toIso8601String(),
        "last_message": '_lastWords',
      },
    );
  });
}

Now app will be working in background mode.

7)Lets now setup Voice listener and bot for replaying,
let say when user say "i want help " or something that contains "help" keyword then system will replied
** "we are sending help"** or user stop listener after saying "Stop".

Now add the following code in same file.



final SpeechToText _speechToText = SpeechToText();
bool _speechEnabled = false;
String _lastWords="Say something";
void _initSpeech() async {
  _speechEnabled = await _speechToText.initialize();

}

void _startListening() async {
  await _speechToText.listen(onResult: _onSpeechResult);
}


void _stopListening() async {
  await _speechToText.stop();
}

Future<void> _onSpeechResult(SpeechRecognitionResult result) async {
  var flutterTts = FlutterTts();
  _lastWords=(result.recognizedWords.toString().toLowerCase());

  if(_lastWords.contains("hello") || _lastWords.contains('help'))
  {

    flutterTts.speak("We are sending help");

  }
  else if(_lastWords.contains('stop'))
    {
      _stopListening();
      flutterTts.speak("Stopped");
    }

}

8)Now lets listen voice in background
add following line in starting of function initializeService()
_initSpeech();
and also add these lines after Timer.periodic function in onStart() Function.
if (_speechEnabled) { _startListening(); }

9)Lets create a UI,in main.dart add the following code:




import 'dart:async';

import 'package:flutter_background_service/flutter_background_service.dart' show AndroidConfiguration, FlutterBackgroundService, IosConfiguration, ServiceInstance;
import 'package:flutter/material.dart';
import 'background_service.dart';
Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await initializeService();

  runApp( const MyApp());
}


class MyApp extends StatefulWidget {

  const MyApp({Key? key,}) : super(key: key);

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  String text = "Stop Service";


  @override
  Widget build(BuildContext context) {
    return  MaterialApp(
      debugShowCheckedModeBanner: false,
      home: Scaffold(

        appBar: AppBar(
          title: const Text("Voice Bot"),
        ),
        body: Center(
          child: Column(
            mainAxisAlignment: MainAxisAlignment.center,
            children: [
              //for listen Continuous change in foreground we will be using Stream builder
              StreamBuilder<Map<String, dynamic>?>(
                stream: FlutterBackgroundService().on('update'),
                  builder: (context,snapshot){
                if (!snapshot.hasData) {
                  return const Center(
                    child: CircularProgressIndicator(),
                  );
                }
                final data = snapshot.data!;
                String? lastMessage = data["last_message"];
                DateTime? date = DateTime.tryParse(data["current_date"]);
                return Column(
                  mainAxisAlignment: MainAxisAlignment.center,
                  children: [
                    Text(lastMessage ?? 'Unknown'),
                    Text(date.toString()),
                  ],
                );
              }),
              Padding(
                padding: const EdgeInsets.all(8.0),
                child: GestureDetector(
                  child: Container(
                    padding: const EdgeInsets.symmetric(vertical: 10,horizontal: 20),
                      decoration: BoxDecoration(
                        color: Colors.blueAccent,
                        borderRadius: BorderRadius.circular(16)
                      ),
                      child: const Text("Foreground Mode",style: TextStyle(
                        color: Colors.white
                      ),)),
                  onTap: () {
                    FlutterBackgroundService().invoke("setAsForeground");

                  },
                ),
              ),
              Padding(
                padding: const EdgeInsets.all(8.0),
                child: GestureDetector(
                  child: Container(
                      padding: const EdgeInsets.symmetric(vertical: 10,horizontal: 20),
                      decoration: BoxDecoration(
                          color: Colors.blueAccent,
                          borderRadius: BorderRadius.circular(16)
                      ),
                      child: const Text("Background Mode",style: TextStyle(
                          color: Colors.white
                      ),)),
                  onTap: () {
                    print('start');
                    FlutterBackgroundService().invoke("setAsBackground");

                  },
                ),
              ),
              Padding(
                padding: const EdgeInsets.all(8.0),
                child: GestureDetector(
                  child: Container(
                    padding: const EdgeInsets.symmetric(vertical: 10,horizontal: 20),
                    decoration: BoxDecoration(
                        color: Colors.blueAccent,
                        borderRadius: BorderRadius.circular(16)
                    ),
                    child: Text(text,style: const TextStyle(
                        color: Colors.white
                    ),),
                  ),
                  onTap: () async {
                    final service=FlutterBackgroundService();

                    var isRunning = await service.isRunning();
                    if (isRunning) {
                      service.invoke("stopService");
                    } else {
                      service.startService();
                    }

                    if (!isRunning) {
                      text = 'Stop Service';
                    } else {
                      text = 'Start Service';
                    }
                    setState(() {});
                  },
                ),
              ),

            ],
          ),
        ),
      ),
    );
  }
}

10)Hurray App done...

Lets compile and have fun.