Mike Wilmer, Roos Bakker and Caspar Meijer at TNO
In the first year of the TrustLLM project, our partner TNO has played an important role in establishing the groundwork for developing trustworthy large language models (LLMs). From sourcing Dutch-language datasets to conducting experiments in factual reasoning, their contributions have laid a strong foundation for the project’s success. This article takes you behind the scenes of TNO’s achievements so far and outlines what lies ahead as we have entered the second year.
Data Sourcing: Building the Foundation
The journey began with an essential step – data. “In the first year, we primarily focused on existing datasets in the Dutch language,” explained Mike Wilmer, Scientist Innovator at TNO, involved in work package 2. “We are currently working with research datasets that have lower requirements. Meanwhile, GPT-NL is working on a higher quality Dutch dataset that meet even stricter requirements. With the first dataset, we can train and learn how to optimize a model. Hopefully, we can eventually proceed with the second dataset” he continued.
Advancing Factual Reasoning
Within the TrustLLM consortium, various work packages focus on different aspects, including factualness, multilinguality, trustworthiness, evaluation, and benchmarking. TNO is mostly involved in the factualness of an LLM.
“In 2024, we explored how well LLMs can recognize facts and their relationships from text,” explained Roos Bakker, AI Research Scientist at TNO. ” We conducted three experiments: testing different prompting methods for extracting knowledge graphs, applying similar methods to ontologies, and evaluating LLMs’ reasoning abilities on a dedicated dataset.”
She continued: ”In short: as facts and relationships grow more complex, LLM performance declines sharply compared to human performance. This year, we’re investigating the extraction of logical rules and axioms, and we plan to continue studying LLM reasoning capabilities.”
These experiments have not only led to a deeper understanding of LLMs’ strengths and limitations but have also resulted in two published research papers, showcasing TNO’s contributions to the broader AI research community.
Looking Ahead: Collaboration and Real-World Applications
With 2025 underway, TNO is concentrating on strengthening partnerships, expanding research, and exploring practical use cases. “This year, I started working with the TrustLLM team on two real-world applications: the Bookbot, which recommends books to children, and the BijsluiterBot, which makes medicine instructions more accessible,” shared Caspar Meijer, Scientist Integrator at TNO. ” We are also evaluating the TrustLLM model against other open-source LLMs. During development, we focus on delivering reusable, high quality software, and I plan to write a short paper on our findings. I’m really looking forward to all of this!”
Beyond technical achievements, TNO remains committed to dissemination, sharing insights to help build a European LLM ecosystem. As the TrustLLM project enters its second year, TNO’s contributions will continue to shape the development of trustworthy AI.