Around six months ago, I pointed to the often overlooked small AI models and EdgeAI. In the fast-paced world of machine intelligence #AI, that feels like an eternity! Since then, we have seen some interesting developments, with specialised modules available for small computers such as the Raspberry Pi. These devices require minimal energy, making them ideal for sustainable tech solutions.
Recently, a quite powerful little module hit the market, boasting 60 TOPS INT8 performance in a tiny M.2 form factor. The SAKURA-II M.2 can be configured with up to 16 GB of LPDDR4X RAM and up to 68 GB/sec memory bandwidth. With this, while being very far from high-end platforms, this is a quite decent performance to run smaller, specialised models. Being a great resource for experimenting with AI and GenAI applications, it also has the potential to offer compact and cost-effective solutions. It will be interesting to see if there will be also some meaningful use cases, e.g. in health care.
Why Does This Matter?
- Accessibility: Small AI models and Edge AI make advanced technology accessible to a broader range of users.
- Sustainability: These solutions are energy-efficient. This makes this technology available with a small energy footprint.
- Innovation: Leveraging open-source models and affordable hardware provides a valuable means of experimenting in unexplored areas, thereby fostering innovation and growth.
