Posts in: AI

A more sovereign AI software setup

I I am experimenting with a "sovereign" software setup that leverages AI technology while strengthening data sovereignty and privacy. This setup combines the power of local frontend tools with advanced AI models hosted in Europe.

Key Components:

1. Local Frontend Tool: I am using AnythingLLM, a versatile tool that allows me to interact with AI models locally. This ensures that key data remains on my device, enhancing privacy and control.

2. OpenAI Compatible API: The frontend tool interfaces seamlessly with an OpenAI-compatible API, making it easy to integrate with various AI services.

3. IONOS AI Model Hub: For the backend, I leveraged the recently introduced IONOS AI Model Hub, hosted in Germany. This ensures that the data I transmit to the model remains within European jurisdiction, complying with stringent data protection laws.

4. European LLM: At the heart of the stack is OpenGPT-X / Teuken-7B, a European, open, multilingual large language model developed by Fraunhofer IAIS. Its a 7B class model, so has seven billion parameters and was trained from scratch with the 24 official languages of the EU.

Why This Setup Matters:

• Data Sovereignty: By keeping data local and within European servers, such a setup ensures compliance with GDPR and other data protection regulations.
• Performance: The combination of a local frontend and a robust backend has the potential to deliver high performance and low latency.
• Innovation: Using European-developed AI models supports local innovation and reduces dependency on foreign technologies.

Future Potential:

This setup is an experiment. The model is not on par with the latest models of global competitors, but as AI continues to evolve, maintaining technological sovereignty will become increasingly important.

By using local tools and European AI models, we can build resilient and compliant AI solutions. There must be sufficient resources for AI research and development in Europe to have a realistic chance of catching up with the best models!

But not every use case needs a huge LLM, so it's also important to understand how this setup works in the real world.

#AI #DataSovereignty #Innovation #EuropeanTech #DigitalTransformation #TechTrends

#FederatedLearning #FL, a decentralised #MachineLearning approach, enables banks to gain insights into customer behaviour without centralising sensitive data. This technology, while challenging to implement, offers personalised services, improved fraud detection, and accurate risk assessment.

While this is good for banks, it also opens interesting perspectives also in #InternationalCooperations handling of data! #FutureOfCooperation

See an example from finance here.

Thanks to Khang Vu Tien for pointing this out!

🔥 #OpenAI is going #OpenSource (sort of) - again. Sam Altman says a major “open weight” model is dropping this summer. Is this the beginning of the end for closed #AI? 👀

Read the full story @ Wired

The weird way an #LLM does the maths - pretty smart-ish!

Never has a technology been so widely adopted yet so mysterious in its inner workings. #Anthropic had to put some serious effort into a method, that gives a glimpse into how their model #Claude „thinks“.

So Anthropic’s deep dive reveals that while the model ends up with the correct answer, its internal process is anything but conventional. It uses quirky, approximate methods that differ starkly from the neat explanation it later provides.

When asked to add 36 and 59, Claude first approximates by summing “40ish and 60ish” and “57ish and 36ish” to arrive at a rough 92ish, then separately focuses on the last digits (6 and 9) to ensure the final number ends in 5. Combining these steps, it correctly computes 95. Pretty smartish. 😅

When asked how it arrived at this result, it gives you a common approach that you can find everywhere online, rather than what it actually did.

So, there’s another reason why it’s best to be careful and check what they say, rather than just believing them.

Full article in technology review

Even more detailed information on more examples in the research paper

This playbook is an inspiring resource with insights into creating impactful #DataScience and #AI #4good projects. It captures DataKind’s strategies and offers valuable lessons to improve the journey. Claiming to be continually updated with new insights and user feedback, it has the potential to become a go-to guide for those diving into #Data4Good initiatives.

🚀 Meet Europe’s new AI Factories: 🇦🇹 AI:AT | 🇧🇬 BRAIN++ | 🇫🇷 AI2F | 🇩🇪 JAIF | 🇵🇱 PIAST | 🇸🇮 SLAIF

With an investment of €485 million, these centres are providing SMEs and start-ups with access to advanced supercomputing, boosting Europe’s AI capacity.

AI Gigafactories, coming soon, will expand Europe’s AI power. Highly relevant! 🌍✨

See full press release.

🚀 Exciting times in #AI development! Showing the potential of #OpenSource! While #DeepSeek as such is open source, it is not yet possible to fully reproduce the model. Now, the #OpenR1 project on #HuggingFace is on a mission to fill the gaps in the DeepSeek-R1 pipeline and make this advanced AI approach accessible to all, allowing anyone to reproduce and build upon it. Work in progress, worth keeping an eye on! 🌐🤝

🔗 Check out the project here

#LLMs: “The worst-case scenario would probably be a dictatorship that writes its propaganda into models on a grand scale”

The phenomenon of “artificial intelligence” is fuelled by the fascination that emanates from a machine that is difficult to distinguish from humans when communicating. Even if human communication is only simulated, people react to this communication as they are used to as social beings.

So here is the thing: Through the training the models had, vertain values and believes are subtlely infused. Using methods from psychology, researchers have found that responses differ according to the gender with which the model is addressed. Political colouring can also be determined with these psychometric tests.

Scientific studies confirm that people are influenced by them. And this already happens through the writing assistants, who complete or change sentences, for example. They are thus part of the writing process and can subtly change the opinions of the writers.

Interesting: large language models LLMs can, for example, help supporters of conspiracy theories to reduce their belief in these theories. But this also harbours the danger that the opposite effect can occur if the models are systematically polluted.

The article in the magazine #ct delves deeper into this dicussion (German, paywall): www.heise.de/select/ct…

🚀 The rise of large language models (#LLMs) and machine intelligence applications comes with significant concerns about the substantial resource requirements of these technologies. The demand for expensive hardware and high energy consumption has sparked creative solutions—one of which is #quantization.

• Quantization reduces the size of large language models (LLMs) by lowering parameter precision (e.g., from 32-bit to 8-bit or 4-bit).
• Benefits:
  • Smaller storage requirements enable deployment on devices like smartphones.
  • Improved energy efficiency and lower computational costs.
• Techniques:
  • Post-training quantization (applied after model training).
  • Quantization-aware training (incorporates quantization during training for better accuracy).
• Challenges: Potential accuracy loss and increased implementation complexity.
• Advances: Research on 1-bit models and methods to avoid matrix multiplications to enhance efficiency.

Quantization is a encouraging approach to making LLMs more accessible and sustainable.

For an insightful explanation of this approach (German, paywall): www.heise.de/hintergru…

🚀 Eyes Wide Open, AIs Wide Open: Staying in Control in the Age of AI 🤖

This talk by Yann Lechelle was given at the recent Open Community Experience (OCX) 2024 in Mainz, Germany, and shed light on Europe’s position in the global AI landscape.

Video link: www.youtube.com/watch

My key takeaways:

• Strategic independence: Emphasized the need for Europe to strengthen its technological sovereignty in AI and reduce its reliance on non-European vendors.
• Open Source & Open Science: Emphasized the central role of open source projects and open science in fostering innovation and ensuring transparency in AI development.
• Unified European Approach: Called for coherent policies and investments to strengthen Europe’s AI capabilities, ensuring that they are consistent with universal values and ethical standards.

tl;dr: Europe must strengthen its AI sovereignty through open source collaboration and unified strategies to remain competitive and uphold its core values.