Intermittent exceptions during concurrent custom-rule execution

[ZhuoIvanLi]

Before submitting a bug report:

• Make sure you are able to repro it on the latest released version
• Perform a quick search for existing issues to check if this bug has already been reported

Steps to reproduce

...\PSScriptAnalyzer-Run.ps1" -Path C:\_work\1\s -Recurse -Settings C:\_work\1\.gdn\rs\psscriptanalyzer\63df5559-3449-4d0b-a8b9-8c97aa98fe3b.ruleset -IgnorePattern .gdn -IncludeDefaultRules -RecurseCustomRulePath -BuildInCustomRules SDLRules -OutputFilePath C:\_work\1\.gdn\.r\psscriptanalyzer\001\psscriptanalyzer.sarif

Expected behavior

Executing PSScriptAnalyzer without any errors

Actual behavior

A customer is intermittently seeing errors from "Invoke-ScriptAnalyzer" while PSScriptAnalyzer runs external SDL rules:

Exception calling "Visit" with "1" argument(s):
"Object reference not set to an instance of an object."

They have also seen:

Exception calling "Visit" with "1" argument(s):
"Collection was modified; enumeration operation may not execute."

The affected rules use a PowerShell class derived from "AstVisitor2" and call "$ScriptBlockAst.Visit($customVisitor)". The problem is intermittent: the same source can fail in one run and pass when rerun. PSScriptAnalyzer reports the exception as a non-terminating error and the process still exits with code 0.

Environment data

Environment: Windows PowerShell 5.1 and PSScriptAnalyzer 1.19.1.

Is this a known concurrency issue in PSScriptAnalyzer’s external-rule execution path or in "Ast.Visit()" under Windows PowerShell 5.1?

[liamjpeters]

It's a known concurrency issue, not specifically with external rule execution - it happens with built-in rules too.

There's a PR that's attempting to tackle (or at least mitigate) this issue to an extent - #2206

[jessehouwing]

It's a known concurrency issue, not specifically with external rule execution - it happens with built-in rules too.

There's a PR that's attempting to tackle (or at least mitigate) this issue to an extent - #2206

The PR actually no longer tries to just mitigate it, but should solve the problem. If you can you can try the PR code and report back, it would be helpful, as the change is quite impactful and getting affirmation it works as intended would expedite the merge process ;).

https://jessehouwing.net/fixing-complex-bugs-with-ai-deepdive-into-powershellscriptanalyzer/#so-please-go-and-break-it