[firebase_ai]: Function calling (tool) doesn't work with liveGenerativeModel #17803
Description
Is there an existing issue for this?
- I have searched the existing issues.
Which plugins are affected?
Other
Which platforms are affected?
No response
Description
I created a liveGenerativeModel
_liveModel = FirebaseAI.googleAI().liveGenerativeModel(
model: 'gemini-2.5-flash-native-audio-preview-09-2025',
liveGenerationConfig: LiveGenerationConfig(
responseModalities: <ResponseModalities>[
ResponseModalities.audio,
],
),
);
And this worked fine until I added the tools like the following:
_liveModel = FirebaseAI.googleAI().liveGenerativeModel(
tools: functionDeclarations != null && functionDeclarations.isNotEmpty
? <Tool>[Tool.functionDeclarations(functionDeclarations)]
: null,
model: 'gemini-2.5-flash-native-audio-preview-09-2025',
liveGenerationConfig: LiveGenerationConfig(
responseModalities: <ResponseModalities>[
ResponseModalities.audio,
],
),
);
the web socket closes as soon as I send the audio stream after starting a new session with the error:
code = 1011
reason = "Internal error occurred"
The documentation says that this model supports Function Calling:
I tried other models like gemini-live-2.5-flash-preview and gemini-2.0-flash-exp, and they work but either they are calling the tool or they respond to the user, they can't do both sequentially as I'm expecting that after they call the tool they reply with LiveServerContent containing audio parts but they don't, and other times they don't use the tool at all even if I provide clear system instructions that tells the model to use the tool with every request. and when they don't use the tool they actually replay with audio. So I'm not sure if these models are dump or is there some inherent limitation that I don't know? and why doesn't gemini-2.5-flash-native-audio-preview-09-2025 work with tools?
Reproducing the issue
Replace the tools here with any tool, I had only one tool
LiveGenerativeModel _liveModel = FirebaseAI.googleAI().liveGenerativeModel(
tools: functionDeclarations != null && functionDeclarations.isNotEmpty
? <Tool>[Tool.functionDeclarations(functionDeclarations)]
: null,
model: 'gemini-2.5-flash-native-audio-preview-09-2025',
liveGenerationConfig: LiveGenerationConfig(
responseModalities: <ResponseModalities>[
ResponseModalities.audio,
],
),
);
Firebase Core version
4.1.1
Flutter Version
3.35.5
Relevant Log Output
Flutter dependencies
Expand Flutter dependencies snippet
Dart SDK 3.9.2
Flutter SDK 3.35.5
sona 1.0.0+1
dependencies:
- auto_route 10.1.2 [flutter path web collection meta]
- autoclose 1.0.4 [async meta]
- autoclose_bloc 1.0.4 [bloc autoclose_flutter]
- autoclose_flutter 1.0.4 [autoclose flutter]
- bloc_concurrency 0.3.0 [bloc stream_transform]
- cloud_firestore 6.0.2 [cloud_firestore_platform_interface cloud_firestore_web collection firebase_core firebase_core_platform_interface flutter meta]
- cupertino_icons 1.0.8
- firebase_ai 3.3.0 [firebase_app_check firebase_auth firebase_core firebase_core_platform_interface flutter http meta web_socket_channel]
- firebase_auth 6.1.0 [firebase_auth_platform_interface firebase_auth_web firebase_core firebase_core_platform_interface flutter meta]
- firebase_core 4.1.1 [firebase_core_platform_interface firebase_core_web flutter meta]
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine]
- flutter_bloc 9.1.1 [bloc flutter provider]
- flutter_localizations 0.0.0 [flutter intl path]
- flutter_pcm_sound 3.3.3 [flutter]
- freezed 3.2.3 [analyzer build build_config collection meta source_gen freezed_annotation json_annotation dart_style pub_semver]
- freezed_annotation 3.1.0 [collection json_annotation meta]
- intl 0.20.2 [clock meta path]
- json_annotation 4.9.0 [meta]
- just_audio 0.10.5 [just_audio_platform_interface just_audio_web audio_session rxdart path path_provider async uuid crypto meta flutter synchronized]
- permission_handler 12.0.1 [flutter meta permission_handler_android permission_handler_apple permission_handler_html permission_handler_windows permission_handler_platform_interface]
- record 6.1.2 [flutter uuid record_platform_interface record_web record_windows record_linux record_android record_ios record_macos]
- talker 5.0.1 [talker_logger]
- uuid 4.5.1 [crypto sprintf meta fixnum]
dev dependencies:
- auto_route_generator 10.2.4 [build source_gen analyzer path build_runner code_builder dart_style glob lean_builder auto_route]
- build_runner 2.7.1 [args async build build_config build_daemon build_runner_core built_collection code_builder crypto dart_style frontend_server_client glob graphs http_multi_server io logging meta mime path pub_semver shelf shelf_web_socket stack_trace stream_transform watcher web_socket_channel yaml]
- flutter_lints 5.0.0 [lints]
- flutter_test 0.0.0 [flutter test_api matcher path fake_async clock stack_trace vector_math leak_tracker_flutter_testing collection meta stream_channel]
- json_serializable 6.11.1 [analyzer async build build_config dart_style json_annotation meta path pub_semver pubspec_parse source_gen source_helper]
transitive dependencies:
- _fe_analyzer_shared 85.0.0 [meta]
- _flutterfire_internals 1.3.62 [collection firebase_core firebase_core_platform_interface flutter meta]
- analyzer 7.6.0 [_fe_analyzer_shared collection convert crypto glob meta package_config path pub_semver source_span watcher yaml]
- ansicolor 2.0.3
- args 2.7.0
- async 2.13.0 [collection meta]
- audio_session 0.2.2 [flutter flutter_web_plugins rxdart meta]
- bloc 9.0.1 [meta]
- boolean_selector 2.1.2 [source_span string_scanner]
- build 3.1.0 [analyzer async build_runner_core convert crypto glob logging package_config path]
- build_config 1.2.0 [checked_yaml json_annotation path pubspec_parse]
- build_daemon 4.0.4 [built_collection built_value crypto http_multi_server logging path pool shelf shelf_web_socket stream_transform watcher web_socket_channel]
- build_resolvers 3.0.3 [analyzer async build build_runner_core collection package_config path pool pub_semver]
- build_runner_core 9.3.1 [analyzer async build build_config build_resolvers build_runner built_collection built_value collection convert crypto glob graphs json_annotation logging meta package_config path pool timing watcher yaml]
- built_collection 5.1.1
- built_value 8.12.0 [built_collection collection fixnum meta]
- characters 1.4.0
- checked_yaml 2.0.4 [json_annotation source_span yaml]
- clock 1.1.2
- cloud_firestore_platform_interface 7.0.2 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface]
- cloud_firestore_web 5.0.2 [_flutterfire_internals cloud_firestore_platform_interface collection firebase_core firebase_core_web flutter flutter_web_plugins]
- code_builder 4.11.0 [built_collection built_value collection matcher meta]
- collection 1.19.1
- convert 3.1.2 [typed_data]
- crypto 3.0.6 [typed_data]
- dart_style 3.1.1 [analyzer args collection package_config path pub_semver source_span yaml]
- fake_async 1.3.3 [clock collection]
- ffi 2.1.4
- file 7.0.1 [meta path]
- firebase_app_check 0.4.1 [firebase_app_check_platform_interface firebase_app_check_web firebase_core firebase_core_platform_interface flutter]
- firebase_app_check_platform_interface 0.2.1 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface]
- firebase_app_check_web 0.2.1 [_flutterfire_internals firebase_app_check_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins web]
- firebase_auth_platform_interface 8.1.2 [_flutterfire_internals collection firebase_core flutter http meta plugin_platform_interface]
- firebase_auth_web 6.0.3 [firebase_auth_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins http_parser meta web]
- firebase_core_platform_interface 6.0.1 [collection flutter flutter_test meta plugin_platform_interface]
- firebase_core_web 3.1.1 [firebase_core_platform_interface flutter flutter_web_plugins meta web]
- fixnum 1.1.1
- flutter_web_plugins 0.0.0 [flutter]
- frontend_server_client 4.0.0 [async path]
- glob 2.1.3 [async collection file path string_scanner]
- graphs 2.3.2 [collection]
- hotreloader 4.3.0 [collection logging path stream_transform vm_service watcher]
- http 1.5.0 [async http_parser meta web]
- http_multi_server 3.2.2 [async]
- http_parser 4.1.2 [collection source_span string_scanner typed_data]
- io 1.0.5 [meta path string_scanner]
- just_audio_platform_interface 4.6.0 [flutter plugin_platform_interface]
- just_audio_web 0.4.16 [just_audio_platform_interface flutter flutter_web_plugins web synchronized]
- leak_tracker 11.0.2 [clock collection meta path vm_service]
- leak_tracker_flutter_testing 3.0.10 [flutter leak_tracker leak_tracker_testing matcher meta]
- leak_tracker_testing 3.0.2 [leak_tracker matcher meta]
- lean_builder 0.1.2 [_fe_analyzer_shared analyzer args xxh3 path collection meta glob watcher dart_style ansicolor source_span stack_trace frontend_server_client yaml hotreloader]
- lints 5.1.1
- logging 1.3.0
- matcher 0.12.17 [async meta stack_trace term_glyph test_api]
- material_color_utilities 0.11.1 [collection]
- meta 1.16.0
- mime 2.0.0
- nested 1.0.0 [flutter]
- package_config 2.2.0 [path]
- path 1.9.1
- path_provider 2.1.5 [flutter path_provider_android path_provider_foundation path_provider_linux path_provider_platform_interface path_provider_windows]
- path_provider_android 2.2.18 [flutter path_provider_platform_interface]
- path_provider_foundation 2.4.2 [flutter path_provider_platform_interface]
- path_provider_linux 2.2.1 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_platform_interface 2.1.2 [flutter platform plugin_platform_interface]
- path_provider_windows 2.3.0 [ffi flutter path path_provider_platform_interface]
- permission_handler_android 13.0.1 [flutter permission_handler_platform_interface]
- permission_handler_apple 9.4.7 [flutter permission_handler_platform_interface]
- permission_handler_html 0.1.3+5 [flutter flutter_web_plugins permission_handler_platform_interface web]
- permission_handler_platform_interface 4.3.0 [flutter meta plugin_platform_interface]
- permission_handler_windows 0.2.1 [flutter permission_handler_platform_interface]
- platform 3.1.6
- plugin_platform_interface 2.1.8 [meta]
- pool 1.5.2 [async stack_trace]
- provider 6.1.5+1 [collection flutter nested]
- pub_semver 2.2.0 [collection]
- pubspec_parse 1.5.0 [checked_yaml collection json_annotation pub_semver yaml]
- record_android 1.4.3 [flutter record_platform_interface]
- record_ios 1.1.4 [flutter record_platform_interface]
- record_linux 1.2.1 [flutter record_platform_interface]
- record_macos 1.1.2 [flutter record_platform_interface]
- record_platform_interface 1.4.0 [flutter plugin_platform_interface]
- record_web 1.2.0 [flutter flutter_web_plugins web record_platform_interface]
- record_windows 1.0.7 [flutter record_platform_interface]
- rxdart 0.28.0
- shelf 1.4.2 [async collection http_parser path stack_trace stream_channel]
- shelf_web_socket 3.0.0 [shelf stream_channel web_socket_channel]
- sky_engine 0.0.0
- source_gen 3.1.0 [analyzer async build dart_style glob path pub_semver source_span yaml]
- source_helper 1.3.8 [analyzer source_gen]
- source_span 1.10.1 [collection path term_glyph]
- sprintf 7.0.0
- stack_trace 1.12.1 [path]
- stream_channel 2.1.4 [async]
- stream_transform 2.1.1
- string_scanner 1.4.1 [source_span]
- synchronized 3.4.0
- talker_logger 5.0.1 [ansicolor web]
- term_glyph 1.2.2
- test_api 0.7.6 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph]
- timing 1.0.2 [json_annotation]
- typed_data 1.4.0 [collection]
- vector_math 2.2.0
- vm_service 15.0.2
- watcher 1.1.4 [async path]
- web 1.1.1
- web_socket 1.0.1 [web]
- web_socket_channel 3.0.3 [async crypto stream_channel web web_socket]
- xdg_directories 1.1.0 [meta path]
- xxh3 1.2.0
- yaml 3.1.3 [collection source_span string_scanner]
Additional context and comments
No response