Useful Sensors’ the AI-In-A-Box module hosts its LLM (large language model) file locally, enabling its inexpensive microprocessor to understand and respond instantly to spoken natural language queries or commands without reference to a data centre.
The module understands natural language, answers queries and solves real-world problems but like other LLM-based AI devices, it operates entirely locally.
The AI module enables intuitive, natural language interaction with electronics devices locally and privately, with no need for an account or internet connection. It can answer queries and solve problems in a similar way to well-known AI tools based on a LLM but uses in-house developed compression and acceleration technologies to host its LLM file locally. This enables the low-cost microprocessor to understand and respond instantly to spoken natural language queries or commands without reference to a data centre.
Disconnected from the internet, the AI-In-A-Box module eliminates user concerns about privacy, snooping, or dependence on third-party cloud services, which are prevalent with conventional LLM-based AI products and services, said Useful Sensors.
Founder and CEO, Pete Warden, explained that the user can talk naturally to the module’s model, with no network connection, account, or API (application programming interface) calls. “It offers the benefits of AI – an intuitive, natural way to control devices in the home, answer queries or just have a chat with a natural-language avatar – without having to share any of your conversations or other data with a big tech company or a cloud service provider,” he said. “At home, every conversation with the box stays within the user’s four walls.”
The AI-In-A-Box module contains a single, general purpose Arm Cortex-A-class microprocessor running all AI operations to answer questions, tell stories and jokes, and give opinions on topics such as art or sport without requiring arrays of high-performance and high-cost GPUs hosted in a data centre, said Useful Sensors.
The module can be used as a bolt-on hardware and software platform for OEMs and developers who want to provide an intuitive and natural voice-based user interface for any electronics product. The software runs locally so the user avoids any need for account registration or system configuration – eliminating privacy concerns.
Other capabilities include real-time closed captions of nearby conversations, which could benefit users with hearing difficulties, for example. Upgrades in development will add the ability to translate between English and other languages, and the ability to use the box as a voice keyboard (speech-to-text function).
The Useful Sensors compressed LLM file stored locally on the AI-In-A-Box module, and its other operating software, are supplied by Useful Sensors as open-source software under a GPL v3 license. Commercial license terms are available for OEMs.
