CONNECTABLE
An AAC communication system that learns how you speak — and drafts the sentence before you finish tapping.
A personal Augmentative and Alternative Communication system for non-verbal users built around a 66-symbol board. A 3-layer AI cascade — bigram lookup, ChromaDB vector search, and a local LLM fallback — predicts what the user wants to say next, and retrains itself nightly from their own phrase history.
The Challenge
Non-verbal users rely on AAC devices to communicate, but most commercial tools are static — they don't learn individual patterns, context, or preferences over time. Users must manually build every phrase from scratch, and generic suggestions rarely match their actual communication needs.
The Solution
ConnectAble combines a 66-symbol communication board with a personalized AI prediction system that learns nightly from the user's phrase history. A 3-layer cascade — bigram lookup, ChromaDB vector search, and LLM fallback — ensures fast, relevant suggestions that improve the more the system is used.
Symbol Communication Board
A 66-symbol grid organized into 5 categories (food, feelings, actions, places, people) lets users build sentences by tapping buttons. The top bar displays real-time AI phrase suggestions that update as each symbol is selected.
3-Layer AI Prediction Cascade
Phrase predictions run through a cascade designed for speed and accuracy. Layer 1 is an instant bigram lookup from a local vocabulary store. Layer 2 does semantic vector search over past phrases via ChromaDB. Layer 3 falls back to Ollama's phi3 LLM when the phrase database is sparse.
Nightly Learning Pipeline
Every night at 2 AM, the system retrains from logged phrase history — rebuilding the bigram map and re-embedding phrases into ChromaDB. The model improves continuously without user intervention.
Text-to-Speech
Supports two TTS modes: fully offline using pyttsx3 / macOS say, or high-quality streaming audio via ElevenLabs for users who need a more natural voice.
Agent Tab
A dedicated agent interface lets users send natural language messages that are classified into intents — make a call, order food, set a reminder, or general chat — and dispatched to tool handlers automatically.
Private by Design
All phrase data is stored locally in an AES-256 encrypted SQLite database (SQLCipher). The LLM runs locally via Ollama. No user data leaves the device unless ElevenLabs TTS is explicitly enabled.
Frontend
React 18 · TypeScript · Vite · Tailwind CSS
Backend
FastAPI · SQLite + AES-256 · Python 3.11+
AI & ML
ChromaDB · sentence-transformers · Ollama phi3
TTS & APIs
pyttsx3 · ElevenLabs · 12 REST endpoints
Cascade Design for Latency
Putting the cheapest operation first (bigram lookup) and the most expensive last (LLM inference) keeps suggestions fast for common phrases while maintaining quality for novel inputs.
Privacy as a Feature
For assistive technology, communication history is highly sensitive. Designing for offline-first with local encryption made privacy a core feature rather than an afterthought.
Personalization Takes Time
The nightly training pipeline means the system improves gradually. Seeding the database with starter phrases was critical for useful predictions from day one.
Graceful Degradation
Building the frontend to work standalone with fallback data when the backend is offline ensured the communication board was always usable, even during development or connectivity issues.