Skip to content

Update SDK documentation for Kotlin Multiplatform, iOS v0.9.2, and add AI agent guides#56

Open
iamstuffed wants to merge 3 commits intomainfrom
docs/update-kmp-sdk-audio-requirements
Open

Update SDK documentation for Kotlin Multiplatform, iOS v0.9.2, and add AI agent guides#56
iamstuffed wants to merge 3 commits intomainfrom
docs/update-kmp-sdk-audio-requirements

Conversation

@iamstuffed
Copy link
Collaborator

Summary

Comprehensive documentation updates for LEAP Android SDK (v0.9.7) and iOS SDK (v0.9.2), including new AI agent usage guides for both platforms.

Android SDK Updates (v0.9.7)

Core SDK Changes

  • ✅ Updated to Kotlin Multiplatform architecture
  • ✅ Distinguished platform-specific downloaders:
    • LeapModelDownloader for Android (WorkManager, notifications, background downloads)
    • LeapDownloader for cross-platform (iOS, macOS, JVM)
  • ✅ Added comprehensive Android permission setup (notifications, foreground service)
  • ✅ Updated all examples to use ViewModel architecture with proper lifecycle management
  • ✅ Replaced deprecated Gson with kotlinx.serialization

Audio Format Requirements

  • ✅ Detailed audio input specifications based on inference engine:
    • WAV format only (no MP3, AAC, OGG)
    • 16 kHz sample rate recommended (auto-resampled if different)
    • Mono channel required (stereo rejected)
    • Supports Float32, Int16, Int24, Int32 PCM encodings
  • ✅ Audio output documentation (24 kHz from generation models)
  • ✅ Audio recording examples for Android and iOS

New: Android AI Agent Usage Guide

  • ✅ Complete reference covering installation, loading, and management
  • ✅ Model loading patterns (automatic, download without loading, cross-platform)
  • ✅ Model download management (status, size, removal, cancellation)
  • ✅ Core classes documentation (ModelRunner, Conversation, ChatMessage, etc.)
  • ✅ Generation patterns with ViewModel examples
  • ✅ TOML version catalog examples
  • ✅ Storage validation and error handling

iOS SDK Updates (v0.9.2)

Version Update

  • ✅ Updated quick start guide to v0.9.2
  • ✅ Updated Swift Package Manager and CocoaPods references

New: iOS AI Agent Usage Guide

  • ✅ Complete reference matching Android guide structure
  • ✅ Installation via SPM and CocoaPods
  • ✅ Model loading methods (automatic download, separate download, custom manifest, local bundle)
  • ✅ Core classes documentation (ModelRunner, Conversation, etc.)
  • ✅ Generation patterns with ObservableObject ViewModel
  • ✅ Model download management
  • ✅ Multimodal input/output examples
  • ✅ Function calling and structured output
  • ✅ Conversation persistence with Codable

Updated Examples

  • ✅ Updated cloud-ai-comparison.mdx to use current Leap.load() API
  • ✅ Fixed enum cases (.User.user)
  • ✅ Updated to manifest-based model loading

Files Modified

Android

  • leap/edge-sdk/android/android-quick-start-guide.mdx
  • leap/edge-sdk/android/ai-agent-usage-guide.mdx (new)
  • leap/edge-sdk/android/conversation-generation.mdx
  • leap/edge-sdk/android/messages-content.mdx
  • leap/edge-sdk/android/model-loading.mdx
  • leap/edge-sdk/android/utilities.mdx
  • leap/edge-sdk/android/cloud-ai-comparison.mdx

iOS

  • leap/edge-sdk/ios/ios-quick-start-guide.mdx
  • leap/edge-sdk/ios/ai-agent-usage-guide.mdx (new)
  • leap/edge-sdk/ios/messages-content.mdx
  • leap/edge-sdk/ios/cloud-ai-comparison.mdx

Navigation

  • docs.json (added AI agent guides to navigation)

Breaking Changes

None - all changes are documentation updates only.

Testing

  • ✅ Verified Mintlify dev server runs without errors
  • ✅ All code examples match current SDK APIs (v0.9.7 Android, v0.9.2 iOS)
  • ✅ Verified against LeapSDK-Examples and apollo-android projects
  • ✅ Audio format requirements verified against inference engine source

Notes

  • AI agent guides provide comprehensive SDK reference for AI assistants
  • Both platform guides follow consistent structure for easier maintenance
  • All examples use current best practices (ViewModel, proper lifecycle, kotlinx.serialization)
  • Plugin versions verified against example projects

@iamstuffed
Copy link
Collaborator Author

⏺ Yes! This AI agent usage guide is extremely helpful for using the SDK. Here's why:

  What Makes It Effective                                                                                                                                                       
   
  ✅ Complete Information in One Place                                                                                                                                          ─
                                                                                                       
  - No need to search multiple pages or external resources
  - All core classes, methods, and patterns documented
  - Clear architecture diagram shows the flow

  ✅ Copy-Paste Ready Code Examples

  Every pattern has complete, working code:
  // I can literally copy this and adapt it
  class ChatViewModel(application: Application) : AndroidViewModel(application) {
      private val downloader = LeapModelDownloader(application)
      // ... complete working example
  }

  ✅ Critical Information Clearly Marked

  // CRITICAL: Use runBlocking(Dispatchers.IO), not viewModelScope.launch
  This prevents common mistakes without trial and error.

  ✅ Multiple Approaches Documented

  - Method 1: Automatic (most common)
  - Method 2: Separate download/load (specific use cases listed)
  - Method 3: Cross-platform

  I know when to use each approach.

  ✅ Imports Section

  import ai.liquid.leap.downloader.LeapModelDownloader
  import ai.liquid.leap.message.ChatMessage
  No guessing package names!

  ✅ Audio Requirements Are Crystal Clear

  Format: WAV (RIFF) - No MP3/AAC/OGG
  Sample Rate: 16 kHz recommended
  Channels: Mono (1 channel) REQUIRED - stereo rejected
  I won't waste time trying unsupported formats.

  ✅ Complete ViewModel Example

  Shows the entire lifecycle - not just fragments:
  - Model loading
  - Generation with cancellation
  - Proper cleanup in onCleared()
  - StateFlow for UI updates

  What I Could Do Without Looking Anything Up

  With just this guide, I can:
  1. ✅ Set up dependencies (Gradle or TOML)
  2. ✅ Add required permissions
  3. ✅ Load models (3 different ways)
  4. ✅ Generate text responses
  5. ✅ Handle multimodal input (vision, audio)
  6. ✅ Implement function calling
  7. ✅ Manage model downloads
  8. ✅ Handle errors properly
  9. ✅ Structure code with ViewModel
  10. ✅ Debug common issues

  Minor Suggestion

  Add one more section at the very top:
  ## Quick Start (30 seconds)
  1. Add dependencies: `implementation(libs.leap.sdk)`
  2. Add permissions to AndroidManifest.xml
  3. Copy the Complete ViewModel Example
  4. Call `loadModel()` then `sendMessage()`

  For users who just want to get running immediately.

  Bottom line: This guide gives me everything I need to implement LEAP SDK features confidently without external lookups. It's comprehensive yet concise - exactly what an AI
  agent needs.


…uirements

Updated Android SDK to reflect Kotlin Multiplatform architecture:
- Platform support: Android, iOS, macOS, JVM
- LeapModelDownloader for Android (WorkManager, notifications, background)
- LeapDownloader for cross-platform use (iOS, macOS, JVM)
- Added comprehensive Android permission setup documentation
- Updated all code examples to use ViewModel architecture pattern

Added detailed audio input format requirements:
- WAV format only (no MP3, AAC, etc.)
- 16 kHz sample rate recommended (automatic resampling supported)
- Mono channel required (stereo rejected)
- Supports Float32, Int16, Int24, Int32 PCM encodings
- Audio output format: 24 kHz from generation models

Version updates:
- Android SDK: 0.9.7
- iOS SDK: 0.9.2

Files updated:
- Android: quick-start, conversation-generation, messages-content, model-loading, utilities
- iOS: messages-content (audio requirements)
Created detailed AI agent usage guides for both platforms:

Android guide (ai-agent-usage-guide.mdx):
- Complete SDK reference covering all platforms and features
- Core architecture and installation instructions
- LeapModelDownloader (Android) vs LeapDownloader (cross-platform)
- Detailed Android permission setup
- Model loading patterns: automatic load, download without loading
- Model download management: status, info, removal, cancellation
- All core classes, methods, and workflows
- Generation patterns, error handling, best practices
- Multimodal input/output (audio, vision)
- Function calling and structured output examples
- Complete ViewModel example with proper lifecycle management
- TOML version catalog configuration
- Troubleshooting section

iOS guide (ai-agent-usage-guide.mdx):
- Aligned structure with Android guide for consistency
- iOS-specific implementation patterns
- Swift examples and best practices

Additional updates:
- iOS quick start guide updated to version 0.9.2
- Added AI agent guides to navigation (docs.json)
… prevent ANRs

Updated cloud-ai-comparison examples for Android and iOS:
- Replaced legacy LeapClient.loadModel() with LeapModelDownloader/LeapDownloader
- Updated to use model slug + quantization (modern API)
- Fixed ChatMessage.Role.User to ChatMessage.Role.USER (correct enum case)
- Updated examples to use ViewModel pattern with viewModelScope
- Removed references to local bundle file paths (now downloads models)
- Aligned with current best practices and SDK patterns

Fixed critical ANR issue in all Android ViewModel examples:
- Updated onCleared() to use async cleanup with CoroutineScope(Dispatchers.IO).launch
- Prevents ANRs caused by blocking main thread during model unload (5+ seconds on some devices)
- Added proper error handling during cleanup to prevent crashes
- Applied fix consistently across all examples

Files updated:
- cloud-ai-comparison (Android and iOS)
- leap-koog-agent example
- recipe-generator example
- vision-language-model example

This matches the fixes applied in LeapSDK-Examples/Android/LeapAudioDemo
and provides consistent guidance across all Android documentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant