The Back Story
There I was, updating one of our CI/CD Bicep deployments. Everything was going smoothly — or so I thought. Part of the deployment involves domain-joining a Windows Server VM to our Active Directory domain. This isn’t some cutting-edge feature or bleeding-edge integration. It’s a good old-fashioned domain join. How hard could it be?
Famous last words.
After watching the deployment fail multiple times, I dove into the logs. Then the VM. Then the extension logs. Then the deployment history. Then the documentation. Then back to the logs again. Repeat that loop for four hours, add in a bit of despair and a lot of coffee, and you’ll get an idea of my afternoon.
Eventually, I figured it out. And since sharing war stories like these is one of the reasons tech blogs exist, I thought I’d write this one up. If this post saves even one person from falling into the same trap, my suffering will not have been in vain.
The Issue
Here’s the setup: we use an Entra ID (formerly Azure AD) service account to perform the domain join.
In this case, the account was named: svc-domainjoin@builtwithcaffeine.cloud
It had a strong, complex password: Lz5BUqz7KtgVPU!GJv^k&yFch7ZGGt5M
Don’t bother trying — it’s a fake account with a fake password. 😏
The Bicep deployment passed this password to the VM using a parameter marked with the @secure() decorator. Totally standard — and smart — practice, because you don’t want secrets showing up in plain text anywhere, especially in deployment logs.
But here’s the gotcha: when the deployment fails — particularly when a domain join extension fails — you’re often left with minimal feedback in the Azure Portal. If the secret you’re passing is marked @secure(), the actual value used is completely hidden from you, even when debugging. You won’t find it in the activity logs, the deployment blade, or the extension logs.
So if you, say, accidentally double-encoded the password, or copied the wrong secret version, or the value contains a special character that gets mangled along the way… well, good luck figuring that out.
And that’s exactly what happened.
The Investigation
As all good engineers the debugging was pretty straight forward, I checked that the account was not locked out, Fine. I checked network connectivity from said machine to the domain controllers, Fine, I even attempted (multiple times) to manually domain join, Fine. - this meant I could rule out, Networking, DNS, Authentication. thus it was something in the pipeline, bicep/arm, Azure portal.
So the next step was to remove the @secure decorator and see what we got in the deployment logs.
As we can see here the complete secret was: Lz5BUqz7KtgVPU!GJv^k&yFch7ZGGt5M and as we can see in the above image that only this part has been passed into the deployment: Lz5BUqz7KtgVPU!GJv^k and this dear reader WAS THE ISSUE OF 4 HOURS! 😭🤪.
The Solutions
Azure DevOps
When passing secrets like passwords into parameters — especially in PowerShell or bash — wrap the value in single quotes (’…’) to treat the content as a literal string. This prevents the shell from misinterpreting special characters like $, &, or “.
Azure DevOps Note
Before running the pipeline, make sure to create a variable group — for example, iacSecrets.
You can reference it in your YAML pipeline using:
variables:
- group: 'iacSecrets'
| |
GitHub Actions
| |
PowerShell
| |
What’s Happening?
superSecurePassword=
This is assigning a value to a variable named superSecurePassword.
""$superSecurePassword""
This is a string that includes the value of the variable $superSecurePassword, but it’s wrapped in escaped double quotes.
"`" is PowerShell’s way to escape a double quote inside a string (the backtick ` is the escape character). $superSecurePassword is the variable whose value you want to include."`" again escapes the closing double quote.
So, if $superSecurePassword was MyPass123, the result would be "MyPass123" (including the quotes).
` (backtick at the end)
This is a line continuation character in PowerShell, meaning the command continues on the next line.
Why This Works
Single quotes (’…’)
Used in bash, Azure DevOps, and GitHub Actions to treat the value as a literal string. This prevents characters like $, &, ^, or ! from being interpreted by the shell.Double quotes with escaping (”…") in PowerShell (pwsh)
In PowerShell Core (pwsh), variables inside double quotes are expanded (interpolated). If your secret includes special characters, you must escape the outer quotes so the entire string is passed as one literal value:
| |
This ensures the variable is wrapped in literal quotes when passed as a parameter, protecting special characters from being split or misinterpreted during command execution.
- Line continuation
The backtick ` is used in PowerShell to continue a command onto the next line, which is helpful for readability and long inline script blocks in CI/CD pipelines.
Wrap Up
This was a classic case of “it’s always the small stuff.” A missing quote, a mangled special character, and suddenly a rock-solid deployment breaks — quietly and frustratingly.
Here’s what to take away:
✅ Wrap secret values in single quotes when passing into bash or YAML to avoid shell misinterpretation.
✅ In PowerShell, escape double quotes to ensure secrets with special characters survive parsing.
✅ Use @secure() in Bicep and ARM templates — but temporarily remove it for debugging when needed.
✅ Don’t trust what you don’t see — if a deployment fails silently, the culprit might be a broken or truncated secret.
✅ Never underestimate the damage one unescaped & or $ can do.
Hopefully, this post saves you hours of debugging and multiple cups of emergency coffee ☕. Happy deploying — and may your secrets stay secret and your pipelines stay green. ✅