Move PS content out of OneDrive #388
Conversation
|
The file should have |
|
|
||
| ```json | ||
| { | ||
| "UserPSContentPath" : "$env:LOCALAPPDATA\\PowerShell\\PSContent", |
There was a problem hiding this comment.
Will the Linux existing CurrentUser modulepaths, etc. be updated to also use this? Will there be a migration approach? I don't like the idea of disparate paths per OS, should follow the XDG standard for both.
https://specifications.freedesktop.org/basedir-spec/latest/
There was a problem hiding this comment.
For Linux I'm thinking there is no need to change their existing paths at all. But I do think they would want this customizability.
So a way to achieve this is to add the current modulepaths to powershell.config.json and make the API retrieve the value every time.
For the migration approach, I'm thinking Linux users won't need to do anything if they keep their current path.
There was a problem hiding this comment.
The issue is that on Linux, for example Modules, it's LOCALAppData\PowerShell by default (which dotnet does environment translation to XDG) and on windows it'll now be PowerShell\PSContent, so it'll be inconsistent, and if someone does this setting in a portable way, they'll have to account for those differences.
Is the PSContent folder really necessary? Why not just put Modules/Scripts/etc. in LocalAppData/PowerShell directly? Then it's the same path on both Windows and Linux (from a dotnet perspective), plus one less level of unnecessary hierarchy. Is there a problem the PSContent folder solves?
There was a problem hiding this comment.
To demonstrate:
Should be the same "path" in dotnet on both, but land differently in the OS. That way if I change my preferred path, it'll still work consistently. I feel it should also be lowercase on both but if you want to add the appropriate handling for windows so it is PowerShell, that's fine.
There was a problem hiding this comment.
+1 for dropping PSContent-subfolder as default. Brings consistency with current structure in Documents/PowerShell and other platforms.
There was a problem hiding this comment.
Looking at the system path we install modules/scripts to a path without a PSContent Folder, so probs best removing this.
On a side note, I think this also means the location of powershell.config has to be hardcoded and cannot be customized.
Nah we allow this to be passed as part of the pwsh executable already, we just default where we look for it in user and system locations. We do expect it to be named that but it can be anything you want.
I personally would like this to be pwsh.config.psd1 or any other file type (like perhaps a .dsc filetype in future 😉😉 @SteveL-MSFT ) instead of json in the future (though should publish a json schema) to stop needing to escape stuff like in filepaths or provide an encrypted file that is decrypted at runtime. (that's all out scope of this RFC though wanted it added so I don't forget it)
There was a problem hiding this comment.
I thought the PSContent sub folder would make it easier to move everything. But I think this can be achieved by addressing each item, Scripts, Modules, Help, and profile, individually.
There was a problem hiding this comment.
We purposely chose to create the PSContent folder because there is already content in $env:LOCALAPPDATA\PowerShell. We don't want the scripts/modules/etc. getting lost among the other files.
There was a problem hiding this comment.
@sdwheeler Do you mean $env:LOCALAPPDATA\Microsoft\PowerShell? That's the folder containing all kinds of cache files. I don't think we use $env:LOCALAPPDATA\PowerShell as of today.
There was a problem hiding this comment.
Yes, I meant $env:LOCALAPPDATA\Microsoft\PowerShell. I thought that is where we intended to put this.
| - Recommendation: Ignore the setting in the machine-level configuration file since this is a user | ||
| setting. No error - just ignore it. |
There was a problem hiding this comment.
Silent errors often lead to confusion. We should warn somewhere (i.e. Event Log, Write-Warning on launch, etc.).
If a system admin sets this expecting all their users to get migrated by default and nothing happens, it'll lead to confusion and frustration.
As a system administrator I may want to set this on a shared system where we would want to avoid users ever dealing with OneDrive issues. We should just match the existing precedent defined in about_PowerShell_Config.
There was a problem hiding this comment.
After Group Policy, settings defined at the AllUsers level take precedence over settings defined for the CurrentUser level.
Are you talking about this precedence? If so you're suggesting we make this be default AllUsers unless specified? I'm not opposed to this but am worried about breaking the current setups since they would be forced to migrate all their UserPSContent to the new location. Any thoughts @sdwheeler ?
There was a problem hiding this comment.
worried about breaking the current setups
While I would tend to agree and caution against breaking existing setups, this would only be the case for admins who deploy this setting retroactively. I would prioritize being consistent in how settings are applied and warn admins what the effect of this on the machine level would have.
If we're not consistent then we get into the, "Well in this scenario, the precedent comes from X, and in that scenario it comes from Y."
There was a problem hiding this comment.
+1 for machine-level preference for consistency and supporting managed/strict environments like shared device and even VDI/VDA where you might place it on a attached user drive.
If a system admin sets this expecting all their users to get migrated by default and nothing happens, it'll lead to confusion and frustration.
Not sure if it's done already, but in general there should be an event logged on startup all config-settings with effective value and source (User, Machine, GroupPolicy) as merge/precedence behavior can get complicated.
There was a problem hiding this comment.
I'm not sure if the following is relevant but I think it'll help with the discussion.
I took another look at the PSModulePath code and as of now machine scoped stuff is not in OneDrive but in program files and not in the documents.
I think we decided to leave this alone and not move anything out of there.
internal static string GetSharedModulePath()
{
#if UNIX
return Platform.SelectProductNameForDirectory(Platform.XDG_Type.SHARED_MODULES);
#else
string sharedModulePath = Environment.GetFolderPath(Environment.SpecialFolder.ProgramFiles);
if (!string.IsNullOrEmpty(sharedModulePath))
{
sharedModulePath = Path.Combine(sharedModulePath, Utils.ModuleDirectory);
}
return sharedModulePath;
#endif
}Will need some clarification from @sdwheeler for this
There was a problem hiding this comment.
Not relevant. Our concern is the user-level modules folder, profile location etc.
My understanding is that a UserPSContentPath setting in the machine pwsh config file would be a mandatory location (template string) for user profile, user-installed modules etc.
In Windows it would be equal to a user configuration in Group Policy, which we can't use here due to cross-platform.
There was a problem hiding this comment.
We need to provide a bypass mechanism for that precedence order, which is out of scope of this RFC's SOW imo
But on a similar note, we need to allow admins a way of testing this on managed devices so perhaps needs to be in this or as a future SOW
| - Recommendation: Ignore the setting in the machine-level configuration file since this is a user | ||
| setting. No error - just ignore it. | ||
|
|
||
| - Will **UserPSContentPath** support environment variables (like `$env:USERNAME` or `%USERNAME%`)? |
There was a problem hiding this comment.
This is required to support the machine-wide config at all without mixing user content, isn't it? env:UserName, $HOME/$env:USERPROFILE, $env:LocalAppData etc.
There was a problem hiding this comment.
I think yes the goal is to support environment variables. I think this would be very useful.
But for machine-wide config I think we are not touching those that are in program files at this time. We are only trying to move the user PSContent folder out of myDocuments
There was a problem hiding this comment.
I hope you are not planning to evaluate the string value as PowerShell script, as that would certainly raise security concerns. Also, that implies a change in the order of processing -- the Runspace needs to be ready before we can handle this key.
I agree it should support environment variables, but we need to parse and replace the env variables like pre-defined variables in ADO YAML (or like in the VSCode mcp setting file), instead of depending on evaluating the whole string value as PowerShell script.
Maybe the syntax for env variables should be like $(env:var-name) (not a good example maybe, still looks quite like PS script :(), to make it a bit clearer to the user that the value is not a PowerShell expression that will be evaluated.
There was a problem hiding this comment.
We should not do what ADO YAML does or reinvent a new syntax like it but we need a suitable cross plat one that is recommended. Perhaps using the ENV:\ PSDrive path could be an option - thoughts on that @daxian-dbw ?
if the PowerShell.Config was a psd1 not json you'd get rid of some of the many headaches that json & Yaml bring us.
We also really should build these files and pre-populate the full path, not do a potentially dangerous lookup operation.
Seperate note - Machine level config should also really move to ProgramData on Windows out of ProgramFiles but is out scope for this RFC
There was a problem hiding this comment.
PowerShell syntax was just a OS-neutral example from my side. My expectation was access to at least some OS system/user/process variables in a cross-platform compatible format, not PowerShell expressions. 👍
Perhaps using the ENV:\ PSDrive path could be an option
IMO this would add to the confusion that it may be resolved in a runspace.
We also really should build these files and pre-populate the full path, not do a potentially dangerous lookup operation
In a ideal world, yes. Docs can have a security recommendation to use static/expanded paths. For user config it could potentially be the only option.
We still need variables for machine-config to have a single value/template that can reference the username and user profile location. The alternatives would be deployment or logon scripts to modify user config for everyone (incl new user profiles) which adds complexity and potentially cost, delays and risk for the user/company.
There was a problem hiding this comment.
@daxian-dbw I was thinking of only supporting a few known variables, not user defined ones. We maybe add to this list in the future?
There was a problem hiding this comment.
@jshigetomi That's fine, but I think the bottom line is that we are not going to evaluate the string value as a PowerShell expression. So, the question is more on how to support those pre-defined variables :)
| - Recommendation: Ignore the setting in the machine-level configuration file since this is a user | ||
| setting. No error - just ignore it. |
There was a problem hiding this comment.
+1 for machine-level preference for consistency and supporting managed/strict environments like shared device and even VDI/VDA where you might place it on a attached user drive.
If a system admin sets this expecting all their users to get migrated by default and nothing happens, it'll lead to confusion and frustration.
Not sure if it's done already, but in general there should be an event logged on startup all config-settings with effective value and source (User, Machine, GroupPolicy) as merge/precedence behavior can get complicated.
|
|
||
| ```json | ||
| { | ||
| "UserPSContentPath" : "$env:LOCALAPPDATA\\PowerShell\\PSContent", |
There was a problem hiding this comment.
+1 for dropping PSContent-subfolder as default. Brings consistency with current structure in Documents/PowerShell and other platforms.
|
|
||
| ## User Experience | ||
|
|
||
| - On startup PowerShell will create a directory in AppData and a configuration file. |
There was a problem hiding this comment.
Will it be created empty or with UserPSContentPath predefined?
This is essential for the machine config discussed below unless machine-level gets precedence (ideally). User-level precedence and potentially predefined setting in user config would block any machine config.
There was a problem hiding this comment.
I'm thinking with the experimental feature turned on UserPSContentPath will be predefined in the config file pointing to the new "recommended" location in LocalAppData. But all of the directories will be empty and the user will have to migrate on their own. As of now for this migration we are thinking about providing an example script.
There was a problem hiding this comment.
Sounds good, as long as the machine config setting with variable support takes precedence. 🙂
There was a problem hiding this comment.
We should also really add this as part of the install options (again after this has been implemented) as to reduce the need for startup to check and create them
| - On startup PowerShell will create a directory in AppData and a configuration file. | |
| - On startup PowerShell will create a directory in AppData and a configuration file if they don't exist. |
There was a problem hiding this comment.
Not sure I follow. It will always have to test the path to cover deleted/renamed config and new user profiles.
|
|
||
| ```json | ||
| { | ||
| "UserPSContentPath" : "$env:LOCALAPPDATA\\PowerShell\\PSContent", |
There was a problem hiding this comment.
+1 on avoiding the <psocntent> subfolder. If the powershell.config.json file doesn't exist, or the key UserPSContentPath is undefined, then we use the default user content path $env:LOCALAPPDATA\powershell. Otherwise, use the user-defined content path.
|
|
||
| ## User Experience | ||
|
|
||
| - On startup PowerShell will create a directory in AppData and a configuration file. |
There was a problem hiding this comment.
Do we create the MyDocument\PowerShell path up on startup today? If so, let's create the env:LocalAppData\powershell folder when starting up; otherwise, let's keep the current behavior.
As for the configuration file, I think there is no need to create it up on start. If the file is missing, we just use the default content location env:LocalAppData\powershell.
There was a problem hiding this comment.
The current behavior is to just point to the location, Install-Module is the one that creates the directory there.
OK will update.
There was a problem hiding this comment.
The reason why we should not create files or folders up on startup unless really necessary is because users may need to run PowerShell in a read-only docker environment.
| - Document the need to keep Modules in the Documents folder to so that SecretManagement | ||
| continues to work for multiple installs of PowerShell 7 (stable and preview). |
There was a problem hiding this comment.
Is this technically possible? There is only one copy of the vault registration, so I guess only 1 registered location of a vault module.
There was a problem hiding this comment.
This is a SOW for Secrets Management & should include a Migration cmdlet as part of that feature, which would align to the document comments.
The last comment can be managed without the use of the docs folder but in line with my other comments about the benefits of using that documents folder already & by having a configurable location you can enable seperate Secret Management settings per version (allowing different operations for different personas like for MSP's etc)
| - Recommendation: Ignore the setting in the machine-level configuration file since this is a user | ||
| setting. No error - just ignore it. | ||
|
|
||
| - Will **UserPSContentPath** support environment variables (like `$env:USERNAME` or `%USERNAME%`)? |
There was a problem hiding this comment.
I hope you are not planning to evaluate the string value as PowerShell script, as that would certainly raise security concerns. Also, that implies a change in the order of processing -- the Runspace needs to be ready before we can handle this key.
I agree it should support environment variables, but we need to parse and replace the env variables like pre-defined variables in ADO YAML (or like in the VSCode mcp setting file), instead of depending on evaluating the whole string value as PowerShell script.
Maybe the syntax for env variables should be like $(env:var-name) (not a good example maybe, still looks quite like PS script :(), to make it a bit clearer to the user that the value is not a PowerShell expression that will be evaluated.
| ```pwsh | ||
| $newPath = "C:\Custom\PowerShell\Modules" | ||
| $currentUserModulePath = [System.Environment]::GetFolderPath('MyDocuments') + "\PowerShell" | ||
| Copy-Item -Path $currentUserModulePath -Destination $newPath -Recurse -Force | ||
| ``` |
There was a problem hiding this comment.
I think you may want to have a script to show users what vault modules are registered today and their hardcoded paths in the registration.
| - PowerShellGet is hardcoded to install scripts and modules in the user's `Documents` folder. It | ||
| will not support this feature. | ||
|
|
||
| ## Implementation questions |
There was a problem hiding this comment.
Can you please lay out all the changes that are needed to PowerShell corresponding to the content folder change? For example, I presume the updateable help needs to be changed to honor the new help content location, but I don't see it mentioned in the RFC.
There was a problem hiding this comment.
So far I see profile, help, scripts, and module paths need to use the new API that looks at the config.json or returns the default location LOCALAPPDATA.
There was a problem hiding this comment.
Other than mostly minor comments, this is a really good RFC, especially as the moving parts here aren't that easy to get your head around, so well done @jshigetomi. Will look forward to the final edits prior to 31st.
I did previously look into all the work needed &thought about raising a RFC myself for this, though also nearly started on a potential solution too & glad I held back instead to give you the chance to do this one.
| C:\Users\UserName\AppData\Local\PowerShell\ | ||
| ├── powershell.config.json (Not Configurable) | ||
| └── <PSContent> (Configurable) | ||
| ├── Scripts (Not Configurable) | ||
| ├── Modules (Not Configurable) | ||
| ├── Help (Not Configurable) | ||
| └── <*profile>.ps1 (Not Configurable) |
There was a problem hiding this comment.
I would want & end up making all of these as configurable options that can make use of either on device/off-device paths like a private gist or fileshare etc
Whilst defaulting to LocalAppData for most is what we want here and makes sense, we should be able to say "actually, I want this in my business or consumer onedrive, or gist or even published to an internal container registry or other location type" & be able to provide this as an arg to pwsh itself without needing to pass a powershell.config.json file.
There are also times when you may want/need to share this with multiple users (like you and your admin user account) and so wouldn't want this in LocalAppData, but actually as part of ProgramData instead or a locked down shared folder, on or off the device.
As a first pass non-configured options makes sense, but longer term goal should be full congifurablility via this being able to be packaged as a new sharable subsytem, managed in powershell.config.json/potentially also in the executable args too.
| - PowerShell currently places profile, modules, and configuration files in the user's Documents | ||
| folder, which is against established conventions for shell configurations and tools. | ||
| - PowerShell content files in OneDrive can lead to unwanted syncing of module files, leading to | ||
| various issues. | ||
| - There is strong community demand for changing this behavior as the current setup is problematic | ||
| for many users. | ||
| - Changing the default location would align PowerShell with other developer tools and improve | ||
| usability. |
There was a problem hiding this comment.
Just so that this is commented as part of this RFC, I'm in the so what camp, especially as PowerShell isn't like other similar tools, so we don't have to stick to those conventions, but also fully understand why we need to do this.
The benefit in keeping as is, is for use with roaming profiles which on *nix isn't a thing in the same way as it is on Windows, but is by users and organisations & especially those not using Cloud Tech and still heavy in on prem installs of AD & other software.
It also goes against giving the user/admin choices, but I do agree we should have a better default, that whilst I don't really think needs to map to other tools, it helps adoption and appeases those coming to PowerShell from those other tools, even if in my opinion there's lots more benefit to using the current set up and allowing the sync with tools like OneDrive, which is something I've used & blogged about using since 2016, as opposed to other tools like ChezMoi (if that's the right spelling)
|
|
||
| ```json | ||
| { | ||
| "UserPSContentPath" : "$env:LOCALAPPDATA\\PowerShell\\PSContent", |
There was a problem hiding this comment.
Looking at the system path we install modules/scripts to a path without a PSContent Folder, so probs best removing this.
On a side note, I think this also means the location of powershell.config has to be hardcoded and cannot be customized.
Nah we allow this to be passed as part of the pwsh executable already, we just default where we look for it in user and system locations. We do expect it to be named that but it can be anything you want.
I personally would like this to be pwsh.config.psd1 or any other file type (like perhaps a .dsc filetype in future 😉😉 @SteveL-MSFT ) instead of json in the future (though should publish a json schema) to stop needing to escape stuff like in filepaths or provide an encrypted file that is decrypted at runtime. (that's all out scope of this RFC though wanted it added so I don't forget it)
| - Document the need to keep Modules in the Documents folder to so that SecretManagement | ||
| continues to work for multiple installs of PowerShell 7 (stable and preview). |
There was a problem hiding this comment.
This is a SOW for Secrets Management & should include a Migration cmdlet as part of that feature, which would align to the document comments.
The last comment can be managed without the use of the docs folder but in line with my other comments about the benefits of using that documents folder already & by having a configurable location you can enable seperate Secret Management settings per version (allowing different operations for different personas like for MSP's etc)
| - Recommendation: Ignore the setting in the machine-level configuration file since this is a user | ||
| setting. No error - just ignore it. |
There was a problem hiding this comment.
We need to provide a bypass mechanism for that precedence order, which is out of scope of this RFC's SOW imo
But on a similar note, we need to allow admins a way of testing this on managed devices so perhaps needs to be in this or as a future SOW
| - Recommendation: Ignore the setting in the machine-level configuration file since this is a user | ||
| setting. No error - just ignore it. | ||
|
|
||
| - Will **UserPSContentPath** support environment variables (like `$env:USERNAME` or `%USERNAME%`)? |
There was a problem hiding this comment.
We should not do what ADO YAML does or reinvent a new syntax like it but we need a suitable cross plat one that is recommended. Perhaps using the ENV:\ PSDrive path could be an option - thoughts on that @daxian-dbw ?
if the PowerShell.Config was a psd1 not json you'd get rid of some of the many headaches that json & Yaml bring us.
We also really should build these files and pre-populate the full path, not do a potentially dangerous lookup operation.
Seperate note - Machine level config should also really move to ProgramData on Windows out of ProgramFiles but is out scope for this RFC
|
Should we lock PowerShell/PowerShell/issues/15552 whilst this RFC is in progress or is it worth keeping open still? |
|
I think that's a good idea, to avoid fragmenting the discussion. |
|
@jshigetomi just a quick pause to thank you for all your hard work on this :) In |
| # PowerShell User Content Location | ||
|
|
||
| This RFC proposes moving the current PowerShell user content location out of OneDrive & | ||
| default to the `AppData` directory on Windows machines. |
There was a problem hiding this comment.
If this is AppData then this would be $env:AppData which is the roaming profile location not the local profile one. Please correct to LocalAppData
| - By default, the PowerShell user content folder will be located in the | ||
| `$env:LOCALAPPDATA\PowerShell`. |
There was a problem hiding this comment.
This needs to be in the Microsoft Folder, on the off chance an organisation wants to fork PowerShell and use their own Org Location alongside use of Microsoft Releases.
| - By default, the PowerShell user content folder will be located in the | |
| `$env:LOCALAPPDATA\PowerShell`. | |
| - By default, the PowerShell user content folder will be located in the | |
| `$env:LOCALAPPDATA\Microsoft\PowerShell`. |
| - The proposed directory structure: | ||
|
|
||
| ``` | ||
| C:\Users\UserName\AppData\Local\PowerShell\ |
There was a problem hiding this comment.
As above
| C:\Users\UserName\AppData\Local\PowerShell\ | |
| C:\Users\UserName\AppData\Local\Microsoft\PowerShell\ |
| **UserPSContentPath** specifies the full path of the content folder. The default value is | ||
| `$env:LOCALAPPDATA\PowerShell`. The user can change this value to a different path. |
There was a problem hiding this comment.
Same again
| **UserPSContentPath** specifies the full path of the content folder. The default value is | |
| `$env:LOCALAPPDATA\PowerShell`. The user can change this value to a different path. | |
| **UserPSContentPath** specifies the full path of the content folder. The default value is | |
| `$env:LOCALAPPDATA\Microsoft\PowerShell`. The user can change this value to a different path. |
|
|
||
| ```json | ||
| { | ||
| "UserPSContentPath" : "$env:LOCALAPPDATA\\PowerShell", |
There was a problem hiding this comment.
Same Again
| "UserPSContentPath" : "$env:LOCALAPPDATA\\PowerShell", | |
| "UserPSContentPath" : "$env:LOCALAPPDATA\\Microsoft\\PowerShell", |
| - The following functionalities will be affected: | ||
| - SecretManagement | ||
| - SecretManagement extension vaults are registered for the current user context in: | ||
| `$env:LOCALAPPDATA\Microsoft\PowerShell\secretmanagement\secretvaultregistry\` |
There was a problem hiding this comment.
This is why we need to keep it in this $env:LOCALAPPDATA\Microsoft\PowerShell\ path as to reduce configuration location sprawl & simplify documentation
| - The following are required changes to PowerShell due to the content folder change: | ||
| - Profile path will need to use the API to get the content folder path. | ||
| - Updateable help path needs to use the API to get the content folder path. | ||
| - Scripts path will need to use the API to get the content folder path. | ||
| - Module path will ned to use the API to get the content folder path. |
There was a problem hiding this comment.
Reduced wording
| - The following are required changes to PowerShell due to the content folder change: | |
| - Profile path will need to use the API to get the content folder path. | |
| - Updateable help path needs to use the API to get the content folder path. | |
| - Scripts path will need to use the API to get the content folder path. | |
| - Module path will ned to use the API to get the content folder path. | |
| - The following Path changes are required in PowerShell due to the content folder change & will be retrived via the API: | |
| - Profile. | |
| - Updateable Help. | |
| - Scripts. | |
| - Modules. |
| - PowerShellGet is hardcoded to install scripts and modules in the user's `Documents` folder. It | ||
| will not support this feature. |
There was a problem hiding this comment.
We may need to recommend regualar checks for resources installed via PowerShellGet
However it also would be a good opportunity for us to actually look at the removal of PowerShellGet & PackageManagement (the sooner the better IMO!)
This however is sadly really a question that is out of scope for this RFC but closely aligned.
This pull request proposes a new RFC (RFC0066) to change the default location of PowerShell user content on Windows machines from OneDrive to the AppData directory. The goal is to align PowerShell with other developer tools and improve usability by preventing unnecessary syncing issues.
Key changes include:
PSModulePath.