High-Performance Computing Center Stuttgart

Making Waves with AI Agents: AI Security through Guardrail Testing and Vulnerability Research

AI agents offer SMEs and start-ups new opportunities to automate workflows, improve customer services and build new products. Connecting agents to business data, tools and services also raises practical security questions: what should you test, how can you assess whether safeguards work, and which risks deserve attention when time and resources are limited?

This session explores two perspectives on AI and cybersecurity: testing agent guardrails in a secure environment and using agentic systems to investigate vulnerabilities across connected services. Through examples from the European AI Hackathon, participants will gain insights into evaluating agent defenses, understanding the evidence behind security findings and asking informed questions when building or selecting AI solutions for their business.

Abstracts

Talk 1: Prompt injections are still the top LLM risk. So where do you safely test your agents?

Prompt injection is still the #1 risk in the OWASP Top 10 for LLM Applications 2026, yet many companies rolling out AI agents are barely aware of it. And you can’t unleash a crowd of hackers on your agents every time you want to test their guardrails, least of all when sensitive data is involved.

In this talk, we show how to make agent security measurable without data leaking out. Building on our prompt injection challenge from last year, we used the European AI Hackathon to build an autonomous Guardrail Arena on the Leonardo supercomputer. Inside this tightly secured environment, unrestricted attacker models probe the agents’ guardrails, abuse monitoring flags suspicious activity, and every detail is recorded for remediation. Our Guardrail Arena lets teams iterate on agent security with real data and turns the results into hard numbers.

Talk 2: More AI agents won’t find more vulnerabilities, but they may become one

AI agents are presented as autonomous vulnerability hunters. What makes them find real, verifiable flaws rather than plausible noise? And what new risks do they create once they are connected to your systems?

Grounded in more than 20 confirmed, authorized cross-surface vulnerability findings, CyberTwin.AI built a multi-agent harness at the European AI Hackathon where attacker, verifier, defender and orchestrator roles share structured evidence, and every finding must pass independent multi-model validation. How agents are organized, how they communicate and how evidence is checked matter far more than how many agents you run. Yet a human expert in the loop is still needed.

Moreover, the agents themselves can become the attack surface. A support agent reading customer emails or a finance agent processing invoices can be hijacked, leak data or take unintended actions, risks that often outweigh classic cross-surface vulnerabilities.

We close with practical takeaways for companies evaluating AI security tools and introduce a future research direction: vulnerability-as-an-agent modeling.

SMEs and start-ups are especially encouraged to join us.

This event is co-organized by HammerHAI and the Pilot-UK AI Factory Antenna at EPCC. 

Location

Online course
Organizer: HLRS, University of Stuttgart, Germany

Start date

Dec 08, 2026
10:00

End date

Dec 08, 2026
12:00

Language

English

Entry level

Basic

Intermediate

Course subject areas

AI / Data in HPC

Topics

Bash (Linux command line basics)

Batch System PBS

HPC System Hunter

Software Modules

Artificial Intelligence

Python

Back to list

Prerequisites and content levels

Prerequisites

  • Basic understanding of large language models and AI agents.
  • Familiarity with cybersecurity concepts such as vulnerabilities and access control.

Content levels

  • Beginners: 30 minutes
  • Intermediate: 1 hour and 30 minutes

Learn more about course curricula and content levels.

Target audience

This session is designed primarily for SMEs, start-ups and industrial companies, including:

  • Business leaders, founders, innovation managers and product owners evaluating AI solutions for their organization.
  • Developers and IT teams building AI-enabled products or connecting agents to business data, tools and services.
  • Cybersecurity professionals and technical consultants helping companies assess agent security and validate safeguards.

Learning outcomes

After this session, participants will be able to:

  • Identify security risks in a business AI use case, including prompt injection, sensitive data exposure and unintended tool actions.
  • Outline guardrail tests with clear success criteria and safeguards for business data before connecting agents to live services.
  • Evaluate security claims from AI vendors or internal teams using test results, recorded evidence and independent validation.
  • Prioritize security fixes and retesting before wider deployment, considering business impact, available resources and differences between test and production environments.

Agenda

  • 10:00–10:15: Introduction 
  • 10:15–10:45: Talk 1: Prompt injections are still the top LLM risk. So where do you safely test your agents? — Klaus Kestel (Lapiscode), Marina Siebold (Codify)
  • 10:45–11:00: Q&A for Talk 1
  • 11:00–11:30: Talk 2: More AI agents won’t find more vulnerabilities, but they may become one — Marin Deresco (CyberTwin.AI)
  • 11:30–11:45: Q&A for Talk 2
  • 11:45–12:00: Discussion and closing

Registration information

Apply for this course via the button at the top of this page.

Registration closes on December 4, 2026.

Late applications may still be accepted depending on course capacity. 

Fees

This course is free of charge.

Only participants from institutions belonging to EU and EuroHPC JU member states can apply to this course.

Contact

Junghwa Lee (HLRS), phone 0711 685 87228, training(at)hlrs.de

Further courses and training team

See the training overview and the Supercomputing Academy pages.
See also information about the HLRS training department and staff.

HLRS Training Collaborations in HPC and AI

HLRS is part of the Gauss Centre for Supercomputing (GCS), together with JSC in Jülich and LRZ in Garching near Munich. SIDE is the German National Competence Centre (NCC) for High-Performance Computing. HLRS is also a member of the Baden-Württemberg initiative bwHPC. 
Since 2025, HLRS has been coordinating one of the AI Factories of the EuroHPC JU: HammerHAI. 

This event is offered as part of HammerHAI, Germany’s first AI Factory, which has a dedicated focus on industry, manufacturing, engineering, and research. HammerHAI provides AI resources and solutions, an upcoming AI-optimized supercomputer, and personalized expert support for AI users at all stages in the AI lifecycle. This project has received funding from the European High Performance Computing Joint Undertaking under grant agreement No. 101234027. This project is co-funded by the European Commission, the German Federal Ministry of Research, Technology and Space (BMFTR), the Baden-Württemberg Ministry of Science, Research and the Arts, the Bavarian State Ministry of Science and the Arts and the Lower Saxony Ministry of Science and Culture.

This project has received funding from the European High Performance Computing Joint Undertaking under grant agreement No. 101263239. This project is co-funded by the European Commission and the UK Government's Department for Science, Innovation and Technology.

Related training
All training