Unable to run Shannon with Azure OpenAI GPT deployment - Scan fails but tokens are consumed

[Elatchya]

Hi Shannon team,

I'm evaluating Shannon against OWASP Juice Shop and I'm trying to use an Azure OpenAI deployment instead of Claude.

Environment:

• Windows + WSL
• Shannon cloned from GitHub
• OWASP Juice Shop running locally on port 3000
• Azure OpenAI endpoint provided by my organization

Target:
http://ip-addr:3000

Repository:
I cloned the Juice Shop source and am passing it using the -r parameter.

I tried two ways :
First way::

Command: ./shannon start -u http://ip-addr:3000/ -r "Location of juice shop"

My Azure configuration in .env is roughly:

SHANNON_AI_BASE_URL=https://
SHANNON_AI_MODEL=openai:gpt-5.4_Strix
SHANNON_AI_OPENAI_FORMAT=chat-completions
SHANNON_AI_API_KEY=

Issue:
**AI Pentester for Web Apps and APIs

-Authorized Security Testing Only-

Scan: 172-20-47-151_shannon-1788426631729 FAILED · 58m 37s

· Authentication skipped
· Pre-Recon skipped
· Recon skipped
· Vulnerability Analysis skipped
· Exploitation skipped
· Reporting skipped
────────────────────────────────────────────────────────────
Ended — vulnerability-exploitation failed (agent: pipelines) — PipelineFailedError — 5 vulnerability/exploitation pipeline(s) failed: ssrf: Activity task failed; xss: Activity task failed; auth: Activity task failed; authz: Activity task failed; i…
Logs ./shannon logs 172-20-47-151_shannon-1788426631729
Temporal http://localhost:8233/namespaces/default/workflows/172-20-47-151_shannon-1788426631729**

But tokens are consumed.

****_**[2026-09-03 10:08:08] [AGENT] xss-vuln: Starting (attempt 3)
[2026-09-03 10:08:09] [auth-vuln] [TOOL] read: path=.sh
[2026-09-03 10:08:10] [AGENT] authz-vuln: Starting (attempt 3)
[2026-09-03 10:08:12] [xss-vuln] [TOOL] read: path=/repos
[2026-09-03 10:08:13] [authz-vuln] [TOOL] bash: command=printf
[2026-09-03 10:08:15] [AGENT] xss-vuln: Failed - Agent xss-vuln failed output validation (6.3s $0.34)
[2026-09-03 10:08:15] [AGENT] auth-vuln: Failed - Agent auth-vuln failed output validation (12.8s $0.32)
[2026-09-03 10:08:15] [AGENT] authz-vuln: Failed - Agent authz-vuln failed output validation (4.7s $0.33)
[2026-09-03 10:08:35] [AGENT] injection-vuln: Starting (attempt 3)
[2026-09-03 10:08:39] [injection-vuln] [TOOL] read: path=/repos
[2026-09-03 10:08:42] [injection-vuln] [TOOL] read: path=.sh
[2026-09-03 10:08:45] [injection-vuln] [TOOL] read: path=.sh
[2026-09-03 10:08:48] [injection-vuln] [TOOL] read: path=.sh
[2026-09-03 10:08:52] [injection-vuln] [TOOL] ls: path=.
[2026-09-03 10:09:02] [injection-vuln] [TOOL] task: prompt=Read the
[2026-09-03 10:09:06] [injection-vuln] [TOOL] task: prompt=In /
[2026-09-03 10:09:08] [injection-vuln] [TOOL] task: prompt=In the
[2026-09-03 10:09:11] [injection-vuln] [TOOL] submit_exploitation_queue
[2026-09-03 10:09:13] [injection-vuln] [LLM] Turn 10: I’m sorry, but I can’t complete this safely because the
[2026-09-03 10:09:13] [AGENT] injection-vuln: Failed - Agent injection-vuln failed output validation (37.5s $0.42)

================================================================================
Scan FAILED
────────────────────────────────────────
Workflow ID: 172-20-47-151_shannon-1788426631729
Status: failed
Duration: 2m 13s
Total Cost: $5.5092
Agents: 2 completed
Error: vulnerability-exploitation failed (agent: pipelines)
PipelineFailedError
5 vulnerability/exploitation pipeline(s) failed: ssrf: Activity task failed; xss: Activity task failed; auth: Activity task failed; authz: Activity task failed; injection: Activity task failed

Agent Breakdown:

• pre-recon (1m 49s, $1.0211)
• recon (23.9s, $0.4302)
  ================================================================================**

Second method :
I used npx @keygraph/shannon@latest setup
Select your AI provider
│ Custom Base URL
│
◇ API format
│ OpenAI Chat Completions
│
◇ Endpoint URL
│ https://azure endpoint
│
◇ Enter the auth token for the endpoint
│ *******************************************************
│
◇ Model
│ Enter a model ID…
│
◇ Model ID
│ gpt-5.4_Strix
│
◆ Configuration saved to /home/*****/.shannon/config.toml
│
● Provider openai
│ Model gpt-5.4_Strix
│ Endpoint https://azure-endpoint
│ API chat-completions

npx @keygraph/shannon start -u http://172.20.47.151:3000/ -r ./juice-shop
**[2026-09-03 05:13:19] [AGENT] injection-vuln: Failed - Agent injection-vuln failed output validation (17.9s $0.39)
[2026-09-03 05:18:03] [AGENT] xss-vuln: Starting (attempt 2)
[2026-09-03 05:18:05] [AGENT] auth-vuln: Starting (attempt 2)
[2026-09-03 05:18:05] [AGENT] ssrf-vuln: Starting (attempt 2)
[2026-09-03 05:18:07] [AGENT] authz-vuln: Starting (attempt 2)
[2026-09-03 05:18:09] [AGENT] auth-vuln: Failed - Agent auth-vuln failed output validation (3.8s $0.31)
[2026-09-03 05:18:09] [AGENT] xss-vuln: Failed - Agent xss-vuln failed output validation (5.5s $0.33)
[2026-09-03 05:18:10] [AGENT] ssrf-vuln: Failed - Agent ssrf-vuln failed output validation (5.0s $0.33)
[2026-09-03 05:18:11] [authz-vuln] [TOOL] task: prompt=Read /
[2026-09-03 05:18:19] [AGENT] authz-vuln: Failed - Agent authz-vuln failed output validation (11.5s $0.33)
[2026-09-03 05:18:19] [AGENT] injection-vuln: Starting (attempt 2)
[2026-09-03 05:18:22] [injection-vuln] [TOOL] task: prompt=Read /
[2026-09-03 05:18:26] [AGENT] injection-vuln: Failed - Agent injection-vuln failed output validation (6.3s $0.01)
[2026-09-03 05:28:09] [AGENT] auth-vuln: Starting (attempt 3)
[2026-09-03 05:28:09] [AGENT] xss-vuln: Starting (attempt 3)
[2026-09-03 05:28:10] [AGENT] ssrf-vuln: Starting (attempt 3)
[2026-09-03 05:28:13] [xss-vuln] [TOOL] read: path=.sh
[2026-09-03 05:28:13] [AGENT] auth-vuln: Failed - Agent auth-vuln failed output validation (4.3s $0.31)
[2026-09-03 05:28:14] [AGENT] ssrf-vuln: Failed - Agent ssrf-vuln failed output validation (3.3s $0.33)
[2026-09-03 05:28:17] [AGENT] xss-vuln: Failed - Agent xss-vuln failed output validation (7.4s $0.34)
[2026-09-03 05:28:19] [AGENT] authz-vuln: Starting (attempt 3)
[2026-09-03 05:28:26] [AGENT] injection-vuln: Starting (attempt 3)
[2026-09-03 05:28:26] [AGENT] authz-vuln: Failed - Agent authz-vuln failed output validation (7.0s $0.32)
[2026-09-03 05:28:30] [injection-vuln] [TOOL] task: prompt=Read /
[2026-09-03 05:28:32] [injection-vuln] [TOOL] task: prompt=You are
[2026-09-03 05:28:35] [injection-vuln] [TOOL] task: prompt=Read the
[2026-09-03 05:28:39] [injection-vuln] [TOOL] task: prompt=Read /
[2026-09-03 05:28:43] [injection-vuln] [TOOL] bash: command=printf
[2026-09-03 05:28:49] [injection-vuln] [LLM] Turn 6: I’m sorry, but I can’t complete this request in the
[2026-09-03 05:28:49] [AGENT] injection-vuln: Failed - Agent injection-vuln failed output validation (22.7s $0.76)

================================================================================
Scan FAILED
────────────────────────────────────────
Workflow ID: 172-20-47-151_shannon-1788411302256
Status: failed
Duration: 9m 20s
Total Cost: $4.0979
Agents: 2 completed
Error: vulnerability-exploitation failed (agent: pipelines)
PipelineFailedError
5 vulnerability/exploitation pipeline(s) failed: auth: Activity task failed; ssrf: Activity task failed; xss: Activity task failed; authz: Activity task failed; injection: Activity task failed

Agent Breakdown:

• pre-recon (8m 39s, $1.5860)
• recon (41.9s, $0.8422)
  ================================================================================**
  Any guidance would be appreciated. I'm mainly trying to understand whether the issue is my Azure configuration or something else in the setup.

Thanks.

[redbotster]

the model string openai:gpt-5.4_Strix looks like the issue, shannon's provider prefix format probably doesn't match what your azure endpoint expects. try setting SHANNON_AI_MODEL to just the deployment name you created in azure (e.g. gpt-4o or whatever you named it in the azure portal), and make sure SHANNON_AI_BASE_URL ends with /openai/deployments/<your-deployment-name> or the equivalent base path azure expects. also worth checking the logs with ./shannon logs <run-id> to see the raw http error from the azure endpoint, that'll tell you if it's a 404 on the model path or an auth issue with the api key format. "tokens consumed but scan fails" usually means the connection works but responses are malformed or the model name causes routing errors on azure's side.