New features in AI LLM request and response handling

Author: AvantMaker

doc/User Guides/2 Basic LLM Chat Implementation.md

@@ -1,5 +1,5 @@
 # ESP32_AI_Connect Library User Guide - 2 Basic LLM Chat Implementation
-> **Version 0.0.3** • Revised: May 10, 2025 • Author: AvantMaker • [https://www.AvantMaker.com](https://www.AvantMaker.com)
+> **Version 0.0.5** • Revised: May 15, 2025 • Author: AvantMaker • [https://www.AvantMaker.com](https://www.AvantMaker.com)
 
 ## Overview
 
@@ -101,11 +101,33 @@ After establishing the WiFi connection, we configure the AI client with specific
 These configuration options allow you to customize the behavior of the AI:
 
 - `setChatTemperature(0.7)`: Controls the randomness/creativity of the responses. Lower values (closer to 0) make responses more deterministic and focused, while higher values (up to 2.0) make them more creative and diverse.
-- `setChatMaxTokens(200)`: Limits the maximum length of the AI's response in tokens (roughly 4 characters per token). This helps control response size and API costs.
+- `setChatMaxTokens(200)`: Sets the maximum number of tokens in the AI's response. This helps regulate response length and manage API costs. Each AI platform handles this parameter slightly differently: OpenAI uses `max_completion_tokens`, Claude uses `max_tokens`, and Gemini uses `maxOutputTokens`. The library automatically adapts to each platform, ensuring compatibility and seamless interaction regardless of which AI provider you choose.
 - `setChatSystemRole("You are a helpful assistant")`: Sets the system message that defines the AI's behavior and personality.
 
 **Note: The parameters set by the above methods are optional. If you do not explicitly configure these parameters, the LLM will use its default values for temperature, max tokens, and system role.**
 
+### Additional Parameter Configuration Methods
+
+The library also provides methods for setting and retrieving custom parameters that are specific to each AI platform:
+
+- `setChatParameters()`: Allows you to set custom parameters in JSON format that are specific to the AI platform you're using. These parameters can include platform-specific options like `top_p`, `presence_penalty`, or any other parameters supported by the platform.
+- `getChatParameters()`: Retrieves the currently set custom parameters.
+
+These methods are demonstrated in the `basic_llm_chat.ino` example in the examples folder. If you need to use platform-specific parameters or want to see how to implement these methods, please refer to that example.
+
+**Important Note**: When using `setChatParameters()`, be aware that:
+1. The parameters must be provided in valid JSON format
+2. If a parameter is already set by a specific method (like `setChatTemperature()`), the value from the specific method will take precedence
+3. The exact parameters available depend on the AI platform you're using
+
+For example, in `basic_llm_chat.ino`, you can see how to use these methods:
+```cpp
+// Set custom parameters
+aiClient.setChatParameters(R"({"top_p":0.95})");
+
+// Get current parameters
+String currentParams = aiClient.getChatParameters();
+```
 
 ## Step 6: Verifying Configuration with Getter Methods
 

examples/basic_example/basic_example.ino

@@ -10,8 +10,8 @@
  * 
  * Author: AvantMaker <admin@avantmaker.com>
  * Author Website: https://www.AvantMaker.com
- * Date: May 7, 2025
- * Version: 1.0.5
+ * Date: May 15, 2025
+ * Version: 1.0.7
  * 
  * Hardware Requirements:
  * - ESP32-based microcontroller (e.g., ESP32 DevKitC, DOIT ESP32 DevKit, etc.)
@@ -30,7 +30,7 @@
  * Repository: https://github.com/AvantMaker/ESP32_AI_Connect
  * 
  * Usage Notes:
- * - Adjust `setChatTemperature`, `setChatMaxTokens`, and `setChatSystemRole` as
+ * - Adjust optional`setChatTemperature`, `setChatMaxTokens`, and `setChatSystemRole` as
  *   needed for your application.
  * - Use getter methods like `getChatTemperature`, `getChatMaxTokens`, and `getChatSystemRole` 
  *   to retrieve current configuration values.
@@ -71,13 +71,14 @@ void setup() {
   Serial.print("IP address: ");
   Serial.println(WiFi.localIP());
 
-  // Configure AI client parameters:
-  aiClient.setChatTemperature(0.7);       // Set response creativity (0.0-2.0)
+  // Configure optional parameters:
+  aiClient.setChatTemperature(0.7);       // Set response creativity (0.0-2.0) 
   aiClient.setChatMaxTokens(200);         // Limit response length (in tokens)
   aiClient.setChatSystemRole("You are a helpful assistant");  // Set assistant behavior
 
-  // Display the configured parameters
-  Serial.println("\nAI Configuration:");
+  // Display the configured parameters set by setChatSystemRole/setChatTemperature/setChatMaxTokens
+  Serial.println("\nDisplay the configured parameters set by");
+  Serial.println("\nsetChatSystemRole / setChatTemperature / setChatMaxTokens:");
   Serial.print("System Role: ");
   Serial.println(aiClient.getChatSystemRole());
   Serial.print("Temperature: ");

examples/basic_llm_chat/basic_llm_chat.ino

@@ -10,8 +10,8 @@
  * 
  * Author: AvantMaker <admin@avantmaker.com>
  * Author Website: https://www.AvantMaker.com
- * Date: May 8, 2025
- * Version: 1.0.3
+ * Date: May 15, 2025
+ * Version: 1.0.7
  * 
  * Hardware Requirements:
  * - ESP32-based microcontroller (e.g., ESP32 DevKitC, DOIT ESP32 DevKit)
@@ -29,7 +29,7 @@
  * Repository: https://github.com/AvantMaker/ESP32_AI_Connect
  * 
  * Usage Notes:
- * - Adjust `setChatSystemRole`, `setChatTemperature`, and `setChatMaxTokens` in `setup()` to customize AI behavior.
+ * - Adjust optional parameters with `setChatSystemRole`, `setChatTemperature`, and `setChatMaxTokens` in `setup()` to customize AI behavior.
  * - Use getter methods like `getChatSystemRole`, `getChatTemperature`, and `getChatMaxTokens` to retrieve current settings.
  * - Enter messages via the Serial Monitor to interact with the AI; responses are displayed with error details if applicable.
  * 
@@ -73,20 +73,30 @@ void setup() {
     while(1) delay(1000); // Halt on failure
   }
 
-  // --- Configure the AI Client ---
+  // --- Configure the AI Client's optional parameters ---
   aiClient.setChatSystemRole("You are a helpful assistant.");
   aiClient.setChatTemperature(0.7); // Set creativity/randomness
   aiClient.setChatMaxTokens(150);   // Limit response length
-  
-  // Display configuration settings
-  Serial.println("\nAI Client Configuration:");
+  // You can set optional custom parameters with setChatParameters()
+  // Note that if a parameter is already set by a method above, it will NOT be overwritten
+  if (aiClient.setChatParameters(R"({"top_p":0.95})")){
+    Serial.println("Request Parameters Set Successfully");
+    Serial.print("Custom Parameters: ");
+    Serial.println(aiClient.getChatParameters());
+  } else {
+    Serial.println("Setting Request Parameters Failed");
+    Serial.println("Error details: " + aiClient.getLastError());
+  }  
+
+// Display the configured parameters set by setChatSystemRole/setChatTemperature/setChatMaxTokens
+  Serial.println("\nDisplay the configured parameters set by");
+  Serial.println("\nsetChatSystemRole / setChatTemperature / setChatMaxTokens:");
   Serial.print("System Role: ");
   Serial.println(aiClient.getChatSystemRole());
   Serial.print("Temperature: ");
   Serial.println(aiClient.getChatTemperature());
   Serial.print("Max Tokens: ");
   Serial.println(aiClient.getChatMaxTokens());
-  Serial.println("Ready to chat!");
 }
 
 void loop() {

examples/custom_llm_chat/custom_llm_chat.ino

@@ -15,8 +15,8 @@
  * 
  * Author: AvantMaker <admin@avantmaker.com>
  * Author Website: https://www.AvantMaker.com
- * Date: May 7, 2025
- * Version: 1.0.1
+ * Date: May 15, 2025
+ * Version: 1.0.3
  * 
  * Hardware Requirements:
  * - ESP32-based microcontroller (e.g., ESP32 DevKitC, DOIT ESP32 DevKit)
@@ -34,7 +34,7 @@
  * Repository: https://github.com/AvantMaker/ESP32_AI_Connect
  * 
  * Usage Notes:
- * - Adjust `setChatSystemRole`, `setChatTemperature`, and `setChatMaxTokens` in `setup()` to customize LLM behavior.
+ * - Adjust optional parameters with `setChatSystemRole`, `setChatTemperature`, and `setChatMaxTokens` in `setup()` to customize LLM behavior.
  * - Use getter methods like `getChatSystemRole`, `getChatTemperature`, and `getChatMaxTokens` to retrieve current settings.
  * - Enter messages via the Serial Monitor to interact with the LLM; responses and errors are displayed.
  * - Verify the custom endpoint URL is correct and accessible for your LLM provider.
@@ -92,16 +92,20 @@ void setup() {
     while(1) delay(1000); // Halt on failure
   }
 
-  // --- Configure the AI Client ---
+  // --- Configure the AI Client's optional parameters ---
   aiClient.setChatSystemRole("You are a helpful assistant.");
   aiClient.setChatTemperature(0.7); // Set creativity/randomness
   aiClient.setChatMaxTokens(150);   // Limit response length
-
+  
   // Print configuration
   Serial.println("\nAI Client Configuration:");
   Serial.println("Platform: " + String(platform));
   Serial.println("Model: " + String(model));
   Serial.println("Custom Endpoint: " + String(customEndpoint));
+
+  // Display the configured parameters set by setChatSystemRole/setChatTemperature/setChatMaxTokens
+  Serial.println("\nDisplay the configured parameters set by");
+  Serial.println("\nsetChatSystemRole / setChatTemperature / setChatMaxTokens:");
   Serial.print("System Role: ");
   Serial.println(aiClient.getChatSystemRole());
   Serial.print("Temperature: ");

examples/tool_calling_demo/tool_calling_demo.ino

@@ -47,6 +47,7 @@
 
 // --- Create the API Client Instance ---
 ESP32_AI_Connect aiClient(platform, apiKey, model);
+// Alternatively, you can use a custom endpoint:
 // ESP32_AI_Connect aiClient(platform, apiKey, model, customEndpoint);
 
 void setup() {
@@ -66,7 +67,7 @@ void setup() {
   Serial.print("IP address: ");
   Serial.println(WiFi.localIP());
 
-  // --- Define Tools for Tool Calling ---
+  // --- Define Tool(s) for Tool Calling ---
   const int numTools = 1;
   String myTools[numTools] = {
     R"({
@@ -94,7 +95,7 @@ void setup() {
   }
   Serial.println(F("Tool calling setup successful."));
 
-  // --- Demonstrate Configuration Methods ---
+  // ---  Configuration Methods ---
   aiClient.setTCChatSystemRole("You are a weather assistant.");// Optional: Set system role message
   aiClient.setTCChatMaxTokens(300);        // Optional: Set maximum tokens for the response
   aiClient.setTCChatToolChoice("auto");   // Optional: Set tool choice mode. 

examples/tool_calling_demo_2/tool_calling_demo_2.ino

@@ -11,8 +11,8 @@
  * 
  * Author: AvantMaker <admin@avantmaker.com>
  * Author Website: https://www.AvantMaker.com
- * Date: May 9, 2025
- * Version: 1.0.3
+ * Date: May 12, 2025
+ * Version: 1.0.5
  * 
  * Hardware Requirements:
  * - ESP32-based microcontroller (e.g., ESP32 DevKitC, DOIT ESP32 DevKit)
@@ -200,7 +200,7 @@ void setup() {
   Serial.println("System Role: " + aiClient.getTCChatSystemRole());
   Serial.println("Max Tokens: " + String(aiClient.getTCChatMaxTokens()));
   Serial.println("Tool Choice: " + aiClient.getTCChatToolChoice());
-  Serial.println("Tool Choice: " + aiClient.getTCRawResponse());
+  Serial.println("AI Raw Response: " + aiClient.getTCRawResponse());
 
   Serial.println("\n--------------------");
   Serial.println("Demo finished. Restart device to run again.");

examples/tool_calling_follow_up_demo/tool_calling_follow_up_demo.ino

@@ -375,13 +375,13 @@ void runToolCallsDemo(String userMessage) {
     String toolResultsJson;
     serializeJson(toolResults, toolResultsJson);
 
-    // --- Demonstrate Configuration Methods for follow-up tool calls request ---
-    // Config the max_token and tool_choice in the follow-up
+    // --- Demonstrate Optional Configuration Methods for follow-up tool calls request ---
+    // Config the token limit (as max_completion_tokens for OpenAI, max_tokens for Claude) and tool_choice in the follow-up
     // request. These parameters are independent of the parameters 
     // set in the initial request. 
 
-    aiClient.setTCReplyMaxTokens(900);
-    aiClient.setTCReplyToolChoice("auto");
+    aiClient.setTCReplyMaxTokens(900); // (Optional) Maximum tokens for the follow-up response
+    aiClient.setTCReplyToolChoice("auto"); // (Optional) Tool choice for the follow-up (can be 
 
     Serial.println("\n---Follow-Up Tool Call Configuration ---");
     Serial.println("Follow-Up Max Tokens: " + String(aiClient.getTCReplyMaxTokens()));
@@ -412,7 +412,7 @@ void runToolCallsDemo(String userMessage) {
     Serial.println("\n--- AI RESPONSE TO TOOL RESULTS ---");
     Serial.println("Finish reason: " + finishReason);
 
-    if (finishReason == "tool_calls") {
+    if (finishReason == "tool_calls" || finishReason == "tool_use") {
       // More tool calls requested - could implement nested calls here
       Serial.println("AI requested more tool calls: " + followUpResult);
       Serial.println("(This example doesn't handle multiple rounds of tool calls)");

examples/tool_calling_follow_up_demo_2/my_info.h

@@ -0,0 +1,7 @@
+// --- User Credentials ---
+const char* ssid = "YOUR_WIFI_SSID";
+const char* password = "YOUR_WIFI_PASSWORD";
+const char* apiKey = "YOUR_API_KEY";  // Your OpenAI API Key
+const char* model = "YOUR_LLM_MODEL"; // Or another model supporting tool calls
+const char* platform = "openai";      // Or "gemini", "openai-compatible" - must match compiled handlers
+// const char* customEndpoint = "YOUR-CUSTOM-ENDPOINT"; // Replace with your custom endpoint