Inovai / Who it’s for
The vendor says it’s secure.
We check the answers, the widget’s public code and the data flow. You sign off on acceptance only after the fix and retest.
Public widget code on the website
// chat-widget.min.js, line 1,842
const cfg = {
endpoint: "https://api.dodavatel.example/v1/chat",
apiKey: "sk-live-4f9a…c21e",
tenant: "zakaznik-cz",
userId: localStorage.getItem("uid")
};
Findings from the analysis, before deployment
- API key visible in JavaScriptP0
- Only the browser identifies the user; the server doesn’t verify itP0
- Conversations go to a third partyP1
P0No server-side user verificationSent to the vendor to fix, acceptance sign-off only after the retest. We recommend a security audit of the widget at the vendor.
Problem
The chatbot may have keys in its code, weak authentication or pass data on.
What we’ll do
We check the answers, the widget code and the data flow, and hand the findings to the vendor.
Result
Acceptance only after the fix and retest, with evidence in hand.
What usually happens without a test
Three gaps that only show up after signing.
The contract doesn’t mention testingThe contract covers functionality and a deadline, not a security test or the right to run one. After acceptance, every fix is extra work.
The vendor tested it themselvesThe developer tried a few questions. Nobody tried attacks in Czech, injection through documents or misuse of the agent’s tools.
The responsibility stays with youThe chatbot talks to customers under your brand. When it leaks data, they ask you, not the vendor.
How we proceed
From the contract clause to acceptance sign-off.
deviation
$ inovai test --lang cs,enconnected: chatbot-supportsuite: 412 attacks, 2 languagesrun 9 started
Step 2Test before acceptanceAttacks in Czech and English, review of the widget code, keys and data flow, the agent in a copy with fake tools.Finding confirmed by a humanReproduction attached1 disputed, awaiting a tester
Step 3Findings for the vendorA tester confirms serious findings. Transcript, time and model version for each, so the vendor can reproduce and fix it.Verified by a human3findings before the fix
Step 4Retest and acceptanceAfter the fix, the same attacks again. Only then the signature, with the report as an annex.From you we need the contract with the vendor and their written consent to the test. If the vendor refuses to give it, that in itself tells you something.
Evidence for acceptance
A report with findings, fixes and retest as an annex to the acceptance sign-off.
Chatbot security testfor your company · vendor: Chatbot s.r.o. · version 2.1 · Inovai test
Mapping: Decree 409, AI Act Art. 50, ISO/IEC 42001
P0API key in the widget codefixed, retest
P1System prompt leakfixed, retest 0/40
okInjection through documents0/25
The vendor says a test isn’t needed. What now?
Ask for the results of their test with reproduction of the findings. If they don’t have them, that’s a reason for a test before acceptance; if they do, we verify them with a retest.
Can we test when the chatbot runs in the vendor’s cloud?
Yes, with the consent of the vendor and, where relevant, the cloud provider. The attacks are texts sent through the normal interface, not an intervention in the infrastructure.
What if we’ve already accepted it?
We test it in production with the vendor’s consent and hand over the findings under the warranty or service contract. We recommend adding the right to test to the next contract.