Settings file creation and validation

[liamjpeters]

PR Summary

Extracted and built upon the settings discoverability of my PR #2134 (omitting all of the JSON file format bits and settings refactoring).

• Adds cmdlet New-ScriptAnalyzerSettingsFile, which generates complete settings files with everything set to it's default value.
• Adds cmdlet Test-ScriptAnalyzerSettingsFile, which validates a settings file (All rule names are valid, all settings valid etc).
• Updates cmdlet Get-ScriptAnalyzerRule to include an Options proprety for configurable rules, including default value.
• Fixes ProcessCustomRulePaths Engine Helper to ignore an empty array of CustomRulePaths.

Motivation

It's not easy or convenient to find out what all the rules and rule settings available are. Your options are really to look at this repo, copy a preset/existing settings file, or look through all of the rules pages on MSLearn. Many people do not even know about settings files.

As a module developer, I want a way to discover what the available rules are and what options they have. I want an effortless way to add a settings file to my project that I can configure to my liking.

New-ScriptAnalyzerSettingsFile

By default this creates a ScriptAnalyzer settings file populated with all rule names and rule configuration (set to default values). The file has some comment help for each field.

The file is always named PSScriptAnalyzerSettings.psd1 so that automatic settings discovery in Invoke-ScriptAnalyzer picks it up without any -Settings parameter needed.

Any rule configuration that is an enum - we attempt to list the possible values as a comment after the default value.

Example output of running New-ScriptAnalyzerSettingsFile in this gist

You can optionally base the rule file on a preset. It takes the settings defined in the preset and normalises it all with the comments and all settings fields.

Example output of running New-ScriptAnalyzerSettingsFile -BaseOnPreset CmdletDesign in this gist.

Note: I'm sure some clever person could eventually plumb this into the VSCode extension, as a code action to create a new settings file perhaps.

Test-ScriptAnalyzerSettingsFile

This validates a settings file and tells you exactly what's wrong and where. It checks for:

• Parse errors
• Unknown rule names in IncludeRules, ExcludeRules, and Rules
• Rules in Rules that aren't configurable
• Unrecognised option names
• Option values that don't match the expected type (bool, int, string, arrays)
• Enum string values (e.g. Kind = 'banana' when the valid values are space and tab)
• Invalid severity values

Each problem comes back as a DiagnosticRecord, unless -Quiet mode used - then just a boolean $true`$false` indicating whether the file is valid or not is returned.

I debated for a while whether this should have just been a rule and after some chicken-or-egg thinking, decided it should be a separate cmdlet.

Get-ScriptAnalyzerRule

Added an Options property to the RuleInfo. I have not updated the format file, so it's not shown by default. I didn't know the implications of updating the format file.

Running Get-ScriptAnalyzerRule -Name PSUseConsistentIndentation | select -expand Options gets you:

Name                OptionType                            DefaultValue PossibleValues
----                ----------                            ------------ --------------
Enable              System.Boolean                               False 
IndentationSize     System.Int32                                     4 
Kind                System.String                                space {Space, Tab}
PipelineIndentation System.String  IncreaseIndentationForFirstPipeline {IncreaseIndentationForFirstPipeline, ...

Successful test run in my fork.

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Change is not breaking
• Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
• Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.