AI Red Teaming for Local RAG Application Security TestingAI Red Teaming for Local RAG Application Security Testing
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Designed and implemented a hands-on AI red teaming environment using Microsoft PyRIT to evaluate the security of a local Retrieval-Augmented Generation application. The project connected a Parrot OS testing environment to a Windows-based AnythingLLM RAG application, with DeepSeek running locally through LM Studio.
I configured PyRIT to communicate with the AnythingLLM workspace through a custom HTTP target and used controlled adversarial objectives to examine how the application handled requests involving financial records, customer information, payroll data, and restricted credentials. The testing workflow included API connectivity validation, isolated Python environment setup, automated prompt execution, response collection, and evidence review.
The lab was built entirely around fictional Kiso Secure business data, allowing realistic sensitive-data disclosure scenarios to be tested without using real customer information or credentials. The resulting workflow demonstrates how automated AI red teaming can be used to identify potential data leakage, retrieval-boundary weaknesses, and unsafe handling of sensitive information in AI-powered applications.
Johnson's avatar
The custom HTTP target that ties PyRIT into the AnythingLLM workspace makes the adversarial testing loop feel very tight.
Stephen's avatar
and it exposes LLM02:2025 Sensitive Information Disclosure
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started