From b2f4f4484c9e6b22d498aa38b8d077f11e454ca6 Mon Sep 17 00:00:00 2001 From: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Date: Thu, 27 Apr 2023 15:50:22 +0400 Subject: [PATCH] Update master with dotnet 6 (#4254) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Draft: Dotnet6 migration (#3879) * Agent on .NET 6 (draft) * Suppress object dispose warnings * Fixed test code analysis * disable RHEL6 * Disable RHEL6, install dotnet 6.0.2 * Disable RHEL6 * fix Winx86 pipeline * Install dotnet x86 * fix dotnet 6 migrations warnings * bump git verstion to 2.36.1 (#3839) Co-authored-by: Andrey Ivanov * Removed NPM module and executable which comes with NodeJS fox non windows distribution (#3833) * Removem NPM module and executable which comes with NodeJS fox non windows distribution * Changed the logic from || and && when removing NPM Co-authored-by: Denis Rumyantsev Co-authored-by: Alexander Smolyakov * Added message - in case when account is not managed, but WindowsLogon… (#3845) * Added message - in case when account is not managed, but WindowsLogonPassword was not specified * Added message - in case when account is not managed, but WindowsLogonPassword was not specified * Fixed typo with condition * Fixed indents Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> * update scripts / rebase * add RHEL6 warning * Agent on .NET 6 (draft) * Suppress object dispose warnings * disable RHEL6 * Disable RHEL6, install dotnet 6.0.2 * fix Winx86 pipeline * Install dotnet x86 * fix dotnet 6 migrations warnings * update azure pipeline to use dotnet 6.0.200 * fix yml indentation * fix windows-x32 pipeline * Update build-job.yml * disable docker detector for legacy containers * disable security checks * disable security checks * Update build-job.yml * Update build-job.yml * Update build-job.yml * update config.sh for net6 support * add native support of the node16 on mac m1 * add osx-arm64 pipeline * fix job name for macos-arm64 * update pipelines for osx-arm64 support * Update build-job.yml * Update build-job.yml * update macos image & disable unit test for osx-arm64 * Update pipeline.yml * switch back to macos-10.15 * set macos image to macos-12 * add osx-arm64 as allowed package * set image to 10.15 * disable falling test for macos * update .NET SDK to 6.0.302 * node16 knob & skip tests on macos * add rhel7 * update pipeline for RHEL7 * update pipeline for rhel7.2 * update pipelines (RHEL7.2) * Update pipeline.yml * update image reference * update runtime framework to 6.0.7 * Update pipeline.yml Co-authored-by: Denis Rumyantsev Co-authored-by: Andrey Ivanov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Max Podriezov Co-authored-by: Denis Rumyantsev Co-authored-by: Alexander Smolyakov Co-authored-by: Anatoly Bolshakov Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> * added logic to skip downgrade if agent have version 3.* and server version is 2.* (#3996) * fix issues after conflict resolving (#4001) * Fix GitHub only release condition (#4002) * BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979) * moved logic to drain queue in separate method "DrainQueues" * Implemented a call of method "DrainQueues" once step is completed * Simplified syntax of property 'JobServerQueue' * Fix condition for onlyGitHubRelease Co-authored-by: Sergey Koryshev * Remove condition from Blob store step * remove branch check on createBranch step * fix pipeline / update agent version * Fixed NullReferenceException in v3.212.0 preview version Agent (#4018) Fix for PublishTelemetry: Added check for taskDefinition.Data.Execution, because it turned out that not all tasks have Execution information. * Update dotnet6-agent from master (#4068) Update dotnet6-agent branch from master * Update releaseNote.md * Fix CA2000 error with secrets masker * update System.IO.FileSystem.AccessControl * Update Common.props (#4109) * Update dotnet6-agent branch from master (#4128) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit af5328b2656c9bf5a70117044a707e011de77083. * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Add fixture for SecretMasker tests * Add fixture for SecretMasker tests * Fail the installation of agent for unsupported OS - RHEL6/CentOS6 * Change Fixtures -> Disposable * git update-index --chmod=+x .\src\dev.sh * Use dispose pattern to fix errors Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve Co-authored-by: Denis Rumyantsev Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Rick Brown Co-authored-by: Konstantin Tyukalov * Merge master to dotnet6-agent branch (#4129) * Merge master to dotnet6-agent branch * git update-index --chmod=+x .\src\dev.sh * Update step name * Change step name * Revert "Allow env.sh to accept additional env var names" (#4144) * Upd dotnet6 agent (#4150) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit af5328b2656c9bf5a70117044a707e011de77083. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit 27e3d4e177e83aac805bf93c38233fa1a9f06101. --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve Co-authored-by: Denis Rumyantsev Co-authored-by: Rick Brown Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk Co-authored-by: Sergey Koryshev Co-authored-by: Konstantin Tyukalov Co-authored-by: Rami Co-authored-by: Martin Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin Co-authored-by: Your Name Co-authored-by: Merlyn Oppenheim Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> * Update minor .net6 version (#4151) * Bump azure pipelines task lib to 4.2.0 (#4156) * Upd dotnet6 agent mar2023 (#4191) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit af5328b2656c9bf5a70117044a707e011de77083. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit 27e3d4e177e83aac805bf93c38233fa1a9f06101. * Bump azure pipelines task lib to 4.2.0 (#4155) * Add warning for deprecated runners (#4162) * Add warning for deprecated runners * Update strings.json * fix typo * Update NodeHandler.cs * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166) * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage * Re-write try-catch * Update TaskRunner.cs * cleanup * Distinguish AzureInstanceMetadata detection to a separate class --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added legacy lib source for support on Ubuntu 22.04 (#4161) * Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176) * Added knob "DrainQueuesAfterTask" with default "false" * move logic to drain queues behind knob DrainQueuesAfterTask * added mocking of GetScopedEnvironment method in step execution context * added mocking of GetScopedEnvironment method in step execution context * changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true * implementd AsBooleanStrict and check in JobRunner * Revert "implementd AsBooleanStrict and check in JobRunner" This reverts commit a1c0c678892a03759543a00b03153adfa0cf5e3b. * Reversed the knob * Add issue templates (#4154) * Add issue templates * Update config.yml * update templates * Update config.yml * Added knob to break pipeline unless user opt-in (#4165) * Added knob to force agent fail if user doesn't opt-in updates on unsupported OS * Resolved PR comments --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Merge remote-tracking branch 'origin/master' into upd_dotnet6_agent_mar2023 # Conflicts: # .azure-pipelines/build-job.yml # .azure-pipelines/pipeline.yml # .vsts.ci.yml # src/Agent.Sdk/Util/PlatformUtil.cs # src/Agent.Worker/JobRunner.cs # src/Agent.Worker/StepsRunner.cs # src/Agent.Worker/TaskRunner.cs # src/Common.props # src/Misc/layoutbin/installdependencies.sh # src/dev.sh * Localization update (#4187) * Added files for onelocbuild * Update localize-pipeline.yml for Azure Pipelines * Fixed typo in resource.resx path * Removed en-US * Update localize-pipeline.yml for Azure Pipelines * LEGO: check in for Localization to temporary branch. (#3246) * LEGO: check in for Localization to temporary branch. (#3268) * LEGO: check in for Localization to temporary branch. (#3265) * LEGO: check in for Localization to temporary branch. (#3279) * Temporary renamed localization folders * Renamed localization folders * Set up schedule and notifications for the localization pipeline (#3269) * [localization] Fixed localization pipeline issue with already localized strings replaced (#3332) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614 Localized file check-in by OneLocBuild Task * LEGO: check in for Localization to temporary branch. (#3363) Co-authored-by: csigs * LEGO: check in for Localization to temporary branch. (#3364) Co-authored-by: csigs * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763 Localized file check-in by OneLocBuild Task * Create PR in OneLocBuild task only on third week of sprint (#3374) * Fix localization pipeline * Add missed change * Added option to disable PR creation * update localize pipeline (#3637) Co-authored-by: Ilya Kuleshov * Localization update (#3612) * Removing Localize folder * RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601) * Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)" This reverts commit 629921e0faea9739aeb926a6c67a443cbf936b96. * Revert "Removing Localize folder" This reverts commit 33cb0ae7c170e9cfb273365c2bc594c6b0ac4da3. Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov * Removed OneLocBuild folder * Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744) * Include strings from Misc/src/layoutbin to LocProject file (#4021) * Update LocProject file * Update CopyOption for LocProject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022) * Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)" This reverts commit 8f5c0c05b3c526d3ac53b9e540917f690445c9ef. * Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023) This reverts commit b9c714bdbaff2a28136e0e5093fe9beef1dc4183. * Include layoutbin into locproject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179) * Update loc strings * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184) * Removing Localize and OneLocBuild folder --------- Co-authored-by: Anatolii Bolshakov (Akvelon INC) Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: KonstantinTyukalov * Add "Mariner" to list of .NET 6 supporting systems (#4197) * Cover unsupported OS failure with FF (#4196) * Cover unsupported OS failure with FF * Update strings.json * Update JobRunner.cs * Update JobRunner.cs * Add knob to enable fetching net6.json file from GitHub (#4200) * Added knob "AGENT_ENABLE_FETCHING_NET6_LIST" * implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems * Removed RuntimeKnobSource for EnableFetchingNet6List * removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6 * fixed typo * Update autoAssignABTT.yml (#4201) * Update AzureInstanceMetadataProvider.cs --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve Co-authored-by: Denis Rumyantsev Co-authored-by: Rick Brown Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk Co-authored-by: Sergey Koryshev Co-authored-by: Konstantin Tyukalov Co-authored-by: Rami Co-authored-by: Martin Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin Co-authored-by: Your Name Co-authored-by: Merlyn Oppenheim Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: Anatolii Bolshakov (Akvelon INC) Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev * Dotnet6 agent merge master (#4243) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit af5328b2656c9bf5a70117044a707e011de77083. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit 27e3d4e177e83aac805bf93c38233fa1a9f06101. * Bump azure pipelines task lib to 4.2.0 (#4155) * Add warning for deprecated runners (#4162) * Add warning for deprecated runners * Update strings.json * fix typo * Update NodeHandler.cs * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166) * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage * Re-write try-catch * Update TaskRunner.cs * cleanup * Distinguish AzureInstanceMetadata detection to a separate class --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added legacy lib source for support on Ubuntu 22.04 (#4161) * Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176) * Added knob "DrainQueuesAfterTask" with default "false" * move logic to drain queues behind knob DrainQueuesAfterTask * added mocking of GetScopedEnvironment method in step execution context * added mocking of GetScopedEnvironment method in step execution context * changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true * implementd AsBooleanStrict and check in JobRunner * Revert "implementd AsBooleanStrict and check in JobRunner" This reverts commit a1c0c678892a03759543a00b03153adfa0cf5e3b. * Reversed the knob * Add issue templates (#4154) * Add issue templates * Update config.yml * update templates * Update config.yml * Added knob to break pipeline unless user opt-in (#4165) * Added knob to force agent fail if user doesn't opt-in updates on unsupported OS * Resolved PR comments --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Localization update (#4187) * Added files for onelocbuild * Update localize-pipeline.yml for Azure Pipelines * Fixed typo in resource.resx path * Removed en-US * Update localize-pipeline.yml for Azure Pipelines * LEGO: check in for Localization to temporary branch. (#3246) * LEGO: check in for Localization to temporary branch. (#3268) * LEGO: check in for Localization to temporary branch. (#3265) * LEGO: check in for Localization to temporary branch. (#3279) * Temporary renamed localization folders * Renamed localization folders * Set up schedule and notifications for the localization pipeline (#3269) * [localization] Fixed localization pipeline issue with already localized strings replaced (#3332) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614 Localized file check-in by OneLocBuild Task * LEGO: check in for Localization to temporary branch. (#3363) Co-authored-by: csigs * LEGO: check in for Localization to temporary branch. (#3364) Co-authored-by: csigs * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763 Localized file check-in by OneLocBuild Task * Create PR in OneLocBuild task only on third week of sprint (#3374) * Fix localization pipeline * Add missed change * Added option to disable PR creation * update localize pipeline (#3637) Co-authored-by: Ilya Kuleshov * Localization update (#3612) * Removing Localize folder * RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601) * Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)" This reverts commit 629921e0faea9739aeb926a6c67a443cbf936b96. * Revert "Removing Localize folder" This reverts commit 33cb0ae7c170e9cfb273365c2bc594c6b0ac4da3. Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov * Removed OneLocBuild folder * Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744) * Include strings from Misc/src/layoutbin to LocProject file (#4021) * Update LocProject file * Update CopyOption for LocProject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022) * Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)" This reverts commit 8f5c0c05b3c526d3ac53b9e540917f690445c9ef. * Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023) This reverts commit b9c714bdbaff2a28136e0e5093fe9beef1dc4183. * Include layoutbin into locproject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179) * Update loc strings * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184) * Removing Localize and OneLocBuild folder --------- Co-authored-by: Anatolii Bolshakov (Akvelon INC) Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: KonstantinTyukalov * Add "Mariner" to list of .NET 6 supporting systems (#4197) * Cover unsupported OS failure with FF (#4196) * Cover unsupported OS failure with FF * Update strings.json * Update JobRunner.cs * Update JobRunner.cs * Add knob to enable fetching net6.json file from GitHub (#4200) * Added knob "AGENT_ENABLE_FETCHING_NET6_LIST" * implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems * Removed RuntimeKnobSource for EnableFetchingNet6List * removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6 * fixed typo * Update autoAssignABTT.yml (#4201) * Fixed typos in strings.json (#3523) * Added trimmed user secret to masker dictionary (#4195) * Add trimmed secret value to masker dictionary * [Refactor] Update MinSecretLength property. (#4137) * [Refactor] Update MinSecretLength property. Removed exception throwing, Added MinSecretLengthLimit public property * Update test MinValue test * Update test * add ability to disable publishing tests metadata (#4209) * moved logic of checking if system supports .NET 6 to JobExtension (#4207) * Increase limit for AZP_IGNORE_SECRETS_SHORTER_THAN knob to 6 (#4214) * Update secret masker length limit * Make property static * Decrease limit to 6 * Update tests * Update tests * Update comment --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Refactor logic to drain queues after each task (#4213) * move draining queues before calling method "Complete" * Revert "BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979)" This reverts commit b3cf2c0e3167288f4b60e0ba38e17b53cd9fa9f5. * implemented logic to drain web console and timeline queues after each task * corrected typo Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * renamed variable "drain" to "shouldDrain" --------- Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * git lfs version upgraded (#4218) * Add noderunner.md (#4202) * Add noderunner.md * Minor fixes * Update noderunner.md * Update noderunner.md * Resolve comments * Minor fixes in noderunner doc * minor fixes * Bug 2030875 - azure-pipelines-task-lib repo logs passwords in plain text (#4223) * added logic to unescape vso command in case there is encoded symbols * Added "TaskCommandHelper" class with method "AddSecret" which checks if decoding percents disabled and adds secret with decoded percents as well * fixed issue with test SetEndpointAuthParameter * moved logic to add decoded secret under check if value is empty * Update AzureInstanceMetadataProvider.cs * Update agentversion --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve Co-authored-by: Denis Rumyantsev Co-authored-by: Rick Brown Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk Co-authored-by: Sergey Koryshev Co-authored-by: Konstantin Tyukalov Co-authored-by: Rami Co-authored-by: Martin Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin Co-authored-by: Your Name Co-authored-by: Merlyn Oppenheim Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: Anatolii Bolshakov (Akvelon INC) Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev Co-authored-by: Rob Hensley * Upd dotnet6 agent before merge with master (#4252) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit af5328b2656c9bf5a70117044a707e011de77083. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit 27e3d4e177e83aac805bf93c38233fa1a9f06101. * Bump azure pipelines task lib to 4.2.0 (#4155) * Add warning for deprecated runners (#4162) * Add warning for deprecated runners * Update strings.json * fix typo * Update NodeHandler.cs * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166) * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage * Re-write try-catch * Update TaskRunner.cs * cleanup * Distinguish AzureInstanceMetadata detection to a separate class --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added legacy lib source for support on Ubuntu 22.04 (#4161) * Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176) * Added knob "DrainQueuesAfterTask" with default "false" * move logic to drain queues behind knob DrainQueuesAfterTask * added mocking of GetScopedEnvironment method in step execution context * added mocking of GetScopedEnvironment method in step execution context * changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true * implementd AsBooleanStrict and check in JobRunner * Revert "implementd AsBooleanStrict and check in JobRunner" This reverts commit a1c0c678892a03759543a00b03153adfa0cf5e3b. * Reversed the knob * Add issue templates (#4154) * Add issue templates * Update config.yml * update templates * Update config.yml * Added knob to break pipeline unless user opt-in (#4165) * Added knob to force agent fail if user doesn't opt-in updates on unsupported OS * Resolved PR comments --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Localization update (#4187) * Added files for onelocbuild * Update localize-pipeline.yml for Azure Pipelines * Fixed typo in resource.resx path * Removed en-US * Update localize-pipeline.yml for Azure Pipelines * LEGO: check in for Localization to temporary branch. (#3246) * LEGO: check in for Localization to temporary branch. (#3268) * LEGO: check in for Localization to temporary branch. (#3265) * LEGO: check in for Localization to temporary branch. (#3279) * Temporary renamed localization folders * Renamed localization folders * Set up schedule and notifications for the localization pipeline (#3269) * [localization] Fixed localization pipeline issue with already localized strings replaced (#3332) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614 Localized file check-in by OneLocBuild Task * LEGO: check in for Localization to temporary branch. (#3363) Co-authored-by: csigs * LEGO: check in for Localization to temporary branch. (#3364) Co-authored-by: csigs * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763 Localized file check-in by OneLocBuild Task * Create PR in OneLocBuild task only on third week of sprint (#3374) * Fix localization pipeline * Add missed change * Added option to disable PR creation * update localize pipeline (#3637) Co-authored-by: Ilya Kuleshov * Localization update (#3612) * Removing Localize folder * RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601) * Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)" This reverts commit 629921e0faea9739aeb926a6c67a443cbf936b96. * Revert "Removing Localize folder" This reverts commit 33cb0ae7c170e9cfb273365c2bc594c6b0ac4da3. Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov * Removed OneLocBuild folder * Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744) * Include strings from Misc/src/layoutbin to LocProject file (#4021) * Update LocProject file * Update CopyOption for LocProject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022) * Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)" This reverts commit 8f5c0c05b3c526d3ac53b9e540917f690445c9ef. * Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023) This reverts commit b9c714bdbaff2a28136e0e5093fe9beef1dc4183. * Include layoutbin into locproject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179) * Update loc strings * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184) * Removing Localize and OneLocBuild folder --------- Co-authored-by: Anatolii Bolshakov (Akvelon INC) Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: KonstantinTyukalov * Add "Mariner" to list of .NET 6 supporting systems (#4197) * Cover unsupported OS failure with FF (#4196) * Cover unsupported OS failure with FF * Update strings.json * Update JobRunner.cs * Update JobRunner.cs * Add knob to enable fetching net6.json file from GitHub (#4200) * Added knob "AGENT_ENABLE_FETCHING_NET6_LIST" * implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems * Removed RuntimeKnobSource for EnableFetchingNet6List * removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6 * fixed typo * Update autoAssignABTT.yml (#4201) * Fixed typos in strings.json (#3523) * Added trimmed user secret to masker dictionary (#4195) * Add trimmed secret value to masker dictionary * [Refactor] Update MinSecretLength property. (#4137) * [Refactor] Update MinSecretLength property. Removed exception throwing, Added MinSecretLengthLimit public property * Update test MinValue test * Update test * add ability to disable publishing tests metadata (#4209) * moved logic of checking if system supports .NET 6 to JobExtension (#4207) * Increase limit for AZP_IGNORE_SECRETS_SHORTER_THAN knob to 6 (#4214) * Update secret masker length limit * Make property static * Decrease limit to 6 * Update tests * Update tests * Update comment --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Refactor logic to drain queues after each task (#4213) * move draining queues before calling method "Complete" * Revert "BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979)" This reverts commit b3cf2c0e3167288f4b60e0ba38e17b53cd9fa9f5. * implemented logic to drain web console and timeline queues after each task * corrected typo Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * renamed variable "drain" to "shouldDrain" --------- Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * git lfs version upgraded (#4218) * Add noderunner.md (#4202) * Add noderunner.md * Minor fixes * Update noderunner.md * Update noderunner.md * Resolve comments * Minor fixes in noderunner doc * minor fixes * Bug 2030875 - azure-pipelines-task-lib repo logs passwords in plain text (#4223) * added logic to unescape vso command in case there is encoded symbols * Added "TaskCommandHelper" class with method "AddSecret" which checks if decoding percents disabled and adds secret with decoded percents as well * fixed issue with test SetEndpointAuthParameter * moved logic to add decoded secret under check if value is empty * Fix: Remove invalid chars from filename (#3831) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Add force of _tasks creation by knob (#4236) * Upgrade vss api netcore version, has BuildXL changes (#4220) * Upgrade vss api netcore version, has BuildXL changes * Adding Nuget authenticate task * Nuget auth task before the build * Try v0 * Remove nuget auth task * Nuget auth * Remove auth task * Remove redundant security package declaration * Adding System.Security.Cryptography.ProtectedData back * Adding cred provider * Removing redundant system.buffers reference * Supress warnings * Remove unused variable * Removing HashType.64k hardcoding * update packages * Update agentversion --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve Co-authored-by: Denis Rumyantsev Co-authored-by: Rick Brown Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk Co-authored-by: Sergey Koryshev Co-authored-by: Konstantin Tyukalov Co-authored-by: Rami Co-authored-by: Martin Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin Co-authored-by: Your Name Co-authored-by: Merlyn Oppenheim Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: Anatolii Bolshakov (Akvelon INC) Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev Co-authored-by: Rob Hensley Co-authored-by: Mihai Ratoiu <89085590+mihaiandreiratoiu@users.noreply.github.com> Co-authored-by: fadnavistanmay <43892780+fadnavistanmay@users.noreply.github.com> * +x src/dev.sh --------- Co-authored-by: Denis Rumyantsev Co-authored-by: Andrey Ivanov Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Max Podriezov Co-authored-by: Denis Rumyantsev Co-authored-by: Alexander Smolyakov Co-authored-by: Anatoly Bolshakov Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Sergey Koryshev Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov Co-authored-by: Liliia Sabitova Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Steve Co-authored-by: Rick Brown Co-authored-by: Eric van Wijk Co-authored-by: Rami Co-authored-by: Martin Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin Co-authored-by: Your Name Co-authored-by: Merlyn Oppenheim Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: csigs Co-authored-by: Egor Bryzgalov Co-authored-by: Anatoly Bolshakov Co-authored-by: csigs Co-authored-by: Nikita Ezzhev Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev Co-authored-by: Rob Hensley Co-authored-by: Mihai Ratoiu <89085590+mihaiandreiratoiu@users.noreply.github.com> Co-authored-by: fadnavistanmay <43892780+fadnavistanmay@users.noreply.github.com> --- .azure-pipelines/build-job.yml | 20 ++++++---- .azure-pipelines/pipeline.yml | 37 +++++++++++++++---- .vsts.ci.yml | 13 +++++-- releaseNote.md | 6 ++- src/Agent.Listener/Agent.Listener.csproj | 2 +- .../WindowsServiceControlManager.cs | 1 + src/Agent.Listener/NuGet.Config | 1 + src/Agent.Listener/SelfUpdater.cs | 6 +++ src/Agent.Plugins/TFCliManager.cs | 16 ++++---- src/Agent.Plugins/TeeCliManager.cs | 18 ++++----- src/Agent.Plugins/TfsVCCliManager.cs | 36 +++++++++--------- src/Agent.Sdk/Agent.Sdk.csproj | 1 + src/Agent.Sdk/Knob/AgentKnobs.cs | 7 ++++ .../Util/AzureInstanceMetadataProvider.cs | 2 +- src/Agent.Worker/Build/TFCommandManager.cs | 18 ++++----- src/Agent.Worker/Build/TeeCommandManager.cs | 18 ++++----- src/Agent.Worker/Build/TfsVCCommandManager.cs | 26 ++++++------- src/Agent.Worker/Handlers/NodeHandler.cs | 8 ++++ src/Agent.Worker/Handlers/StepHost.cs | 2 +- src/Agent.Worker/JobRunner.cs | 5 ++- src/Agent.Worker/Variables.cs | 5 +-- src/Common.props | 5 ++- ...crosoft.VisualStudio.Services.Agent.csproj | 1 + .../Util/VarUtil.cs | 2 +- src/Misc/dotnet-install.sh | 0 src/Misc/externals.sh | 20 +++++++++- src/Misc/layoutbin/installdependencies.sh | 31 +++++----------- src/Misc/layoutroot/config.sh | 37 +++++++++++-------- src/Misc/layoutroot/env.sh | 16 ++++---- src/Test/L0/ConstantGenerationL0.cs | 4 +- src/Test/L0/HostContextExtensionL0.cs | 2 + src/Test/L0/Listener/AgentL0.cs | 2 + .../AgentCapabilitiesProviderTestL0.cs | 8 ++-- src/Test/L0/Listener/JobDispatcherL0.cs | 6 +++ src/Test/L0/Plugin/ChunkerTests.cs | 1 + src/Test/L0/Plugin/FingerprintCreatorTests.cs | 2 + src/Test/L0/Plugin/LogPluginHostL0.cs | 6 +-- .../TestFileShareProviderL0.cs | 5 ++- .../TestResultParser/EnumerableExtensionL0.cs | 2 + src/Test/L0/ProcessExtensionL0.cs | 3 ++ src/Test/L0/TestHostContext.cs | 2 + src/Test/L0/Util/IOUtilL0.cs | 2 + src/Test/L1/Mock/FakeConfigurationStore.cs | 4 +- src/agentversion | 2 +- src/dev.sh | 13 ++++++- tools/CredScanRegexes/readme.md | 2 +- 46 files changed, 265 insertions(+), 161 deletions(-) mode change 100755 => 100644 src/Misc/dotnet-install.sh mode change 100755 => 100644 src/Misc/externals.sh mode change 100755 => 100644 src/Misc/layoutbin/installdependencies.sh diff --git a/.azure-pipelines/build-job.yml b/.azure-pipelines/build-job.yml index 90e5f5bd76..5a86bf4152 100644 --- a/.azure-pipelines/build-job.yml +++ b/.azure-pipelines/build-job.yml @@ -25,6 +25,7 @@ parameters: - linux - osx - rhel.6 + - rhel.7.2 - name: arch type: string @@ -83,6 +84,9 @@ jobs: ${{ if ne(parameters.os, 'win') }}: devCommand: ./dev.sh layoutRoot: '_layout/${{ parameters.os }}-${{ parameters.arch }}' + DisableCFSDetector: true + DisableDockerDetector: true + NugetSecurityAnalysisWarningLevel: "none" ${{ if eq(parameters.os, 'rhel.6') }}: AGENT_USE_NODE10: true @@ -104,7 +108,7 @@ jobs: branch: ${{ parameters.branch }} # 1ES images used on the ARM pool doesn't contain unzip tool, so we need to install it before starting the build - - ${{ if eq(parameters.arch, 'arm64') }}: + - ${{ if and(eq(parameters.arch, 'arm64'), ne(parameters.os, 'osx')) }}: - script: sudo apt-get update && sudo apt-get -y install unzip displayName: Install unzip @@ -112,6 +116,7 @@ jobs: - script: ${{ variables.devCommand }} layout Release ${{ parameters.os }}-${{ parameters.arch }} workingDirectory: src displayName: Build & Layout Release + retryCountOnTaskFailure: 5 ${{ if parameters.enableADOLogIssue }}: env: ADO_ENABLE_LOGISSUE: true @@ -126,9 +131,9 @@ jobs: - ${{ if parameters.unitTests }}: - ${{ if and(eq(parameters.os, 'win'), eq(parameters.arch, 'x86')) }}: - task: UseDotNet@2 - displayName: Install .NET Core 3.1 SDK + displayName: Install .NET Core 6 Runtime inputs: - version: '3.1.x' + version: '6.0.x' packageType: 'runtime' installationPath: 'C:\Program Files (x86)\dotnet' env: @@ -146,12 +151,11 @@ jobs: - ${{ if parameters.functionalTests }}: - ${{ if and(eq(parameters.os, 'win'), eq(parameters.arch, 'x86')) }}: - task: UseDotNet@2 - displayName: Install .NET Core 3.1 SDK + displayName: Install .NET Core 6 SDK inputs: - version: '3.1.x' - packageType: runtime - performMultiLevelLookup: true - installationPath: 'C:\Program Files (x86)\dotnet' + version: '6.0.405' + packageType: sdk + performMultiLevelLookup: true - script: ${{ variables.devCommand }} testl1 Debug ${{ parameters.os }}-${{ parameters.arch }} workingDirectory: src displayName: Functional tests diff --git a/.azure-pipelines/pipeline.yml b/.azure-pipelines/pipeline.yml index 3ca7d40d02..0d9e4dc75d 100644 --- a/.azure-pipelines/pipeline.yml +++ b/.azure-pipelines/pipeline.yml @@ -46,12 +46,15 @@ parameters: - name: linux_arm64 type: boolean default: true -- name: rhel6_x64 +- name: rhel7_x64 type: boolean default: true - name: macOS_x64 type: boolean default: true +- name: macOS_arm64 + type: boolean + default: true resources: containers: @@ -59,6 +62,8 @@ resources: image: azpagentinfra/arm:latest - container: dotnetcore_centos6 image: azpagentinfra/centos6:latest + - container: dotnetcore_centos7 + image: azpagentinfra/centos7:latest stages: @@ -163,16 +168,16 @@ stages: publishArtifacts: ${{ parameters.publishArtifacts }} buildAlternatePackage: ${{ parameters.buildAlternatePackage }} - # RHEL6 (x64) - - ${{ if and(parameters.rhel6_x64, not(parameters.skipRhelRelease), parameters.releaseRhelOnly) }}: + # RHEL7 (x64) + - ${{ if parameters.rhel7_x64 }}: - template: build-jobs.yml parameters: - jobName: build_rhel6_x64 - displayName: RHEL6 (x64) + jobName: build_rhel7_x64 + displayName: RHEL7 (x64) pool: vmImage: ubuntu-20.04 - container: dotnetcore_centos6 - os: rhel.6 + container: dotnetcore_centos7 + os: linux arch: x64 branch: ${{ parameters.branch }} componentDetection: false @@ -200,4 +205,22 @@ stages: publishArtifacts: ${{ parameters.publishArtifacts }} buildAlternatePackage: ${{ parameters.buildAlternatePackage }} + # macOS arm64 + - ${{ if parameters.macOS_arm64 }}: + - template: build-jobs.yml + parameters: + jobName: build_osx_arm64 + displayName: macOS (arm64) + pool: + vmImage: macOS-12 + os: osx + arch: arm64 + branch: ${{ parameters.branch }} + componentDetection: ${{ parameters.componentDetection }} + unitTests: false + functionalTests: false + sign: ${{ parameters.sign }} + publishArtifacts: ${{ parameters.publishArtifacts }} + buildAlternatePackage: ${{ parameters.buildAlternatePackage }} + - ${{ parameters.postBuildStages }} diff --git a/.vsts.ci.yml b/.vsts.ci.yml index a7e3c7d5ae..9917e704df 100644 --- a/.vsts.ci.yml +++ b/.vsts.ci.yml @@ -19,14 +19,18 @@ parameters: type: boolean displayName: Linux (ARM64) default: false -- name: rhel6_x64 +- name: rhel7_x64 type: boolean - displayName: RHEL 6 (x64) - default: true + displayName: RHEL 7 (x64) + default: false - name: macOS_x64 type: boolean displayName: macOS (x64) default: true +- name: macOS_arm64 + type: boolean + displayName: macOS (arm64) + default: true - name: skipRhelRelease type: boolean default: true @@ -51,5 +55,6 @@ extends: linux_x64: ${{ parameters.linux_x64 }} linux_arm: ${{ parameters.linux_arm }} linux_arm64: ${{ parameters.linux_arm64 }} - rhel6_x64: ${{ parameters.rhel6_x64 }} + rhel7_x64: ${{ parameters.rhel7_x64 }} macOS_x64: ${{ parameters.macOS_x64 }} + macOS_arm64: ${{ parameters.macOS_arm64 }} diff --git a/releaseNote.md b/releaseNote.md index e50c7a3ce0..8db5c25573 100644 --- a/releaseNote.md +++ b/releaseNote.md @@ -5,7 +5,8 @@ | ----------- | ------- | ------- | | Windows x64 | [vsts-agent-win-x64-.zip](https://vstsagentpackage.azureedge.net/agent//vsts-agent-win-x64-.zip) | | | Windows x86 | [vsts-agent-win-x86-.zip](https://vstsagentpackage.azureedge.net/agent//vsts-agent-win-x86-.zip) | | -| macOS | [vsts-agent-osx-x64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//vsts-agent-osx-x64-.tar.gz) | | +| macOS x64 | [vsts-agent-osx-x64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//vsts-agent-osx-x64-.tar.gz) | | +| macOS arm64 | [vsts-agent-osx-arm64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//vsts-agent-osx-arm64-.tar.gz) | | | Linux x64 | [vsts-agent-linux-x64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//vsts-agent-linux-x64-.tar.gz) | | | Linux ARM | [vsts-agent-linux-arm-.tar.gz](https://vstsagentpackage.azureedge.net/agent//vsts-agent-linux-arm-.tar.gz) | | | Linux ARM64 | [vsts-agent-linux-arm64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//vsts-agent-linux-arm64-.tar.gz) | | @@ -71,7 +72,8 @@ See [notes](docs/node6.md) on Node version support for more details. | ----------- | ------- | ------- | | Windows x64 | [pipelines-agent-win-x64-.zip](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-win-x64-.zip) | | | Windows x86 | [pipelines-agent-win-x86-.zip](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-win-x86-.zip) | | -| macOS | [pipelines-agent-osx-x64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-osx-x64-.tar.gz) | | +| macOS x64 | [pipelines-agent-osx-x64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-osx-x64-.tar.gz) | | +| macOS arm64 | [pipelines-agent-osx-arm64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-osx-x64-.tar.gz) | | | Linux x64 | [pipelines-agent-linux-x64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-linux-x64-.tar.gz) | | | Linux ARM | [pipelines-agent-linux-arm-.tar.gz](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-linux-arm-.tar.gz) | | | Linux ARM64 | [pipelines-agent-linux-arm64-.tar.gz](https://vstsagentpackage.azureedge.net/agent//pipelines-agent-linux-arm64-.tar.gz) | | diff --git a/src/Agent.Listener/Agent.Listener.csproj b/src/Agent.Listener/Agent.Listener.csproj index e9e1eb1493..e9d2241960 100644 --- a/src/Agent.Listener/Agent.Listener.csproj +++ b/src/Agent.Listener/Agent.Listener.csproj @@ -16,7 +16,7 @@ - + diff --git a/src/Agent.Listener/Configuration.Windows/WindowsServiceControlManager.cs b/src/Agent.Listener/Configuration.Windows/WindowsServiceControlManager.cs index b90039e5c1..219548e856 100644 --- a/src/Agent.Listener/Configuration.Windows/WindowsServiceControlManager.cs +++ b/src/Agent.Listener/Configuration.Windows/WindowsServiceControlManager.cs @@ -77,6 +77,7 @@ public void ConfigureService(AgentSettings settings, CommandSettings command) { logonPassword = command.GetWindowsLogonPassword(logonAccount); } + catch (ArgumentException exception) { Trace.Warning($"LogonAccount {logonAccount} is not managed service account, although you did not specify WindowsLogonPassword - maybe you wanted to use managed service account? Please see https://aka.ms/gmsa for guidelines to set up sMSA/gMSA account. "); diff --git a/src/Agent.Listener/NuGet.Config b/src/Agent.Listener/NuGet.Config index b62d806594..223189bf17 100644 --- a/src/Agent.Listener/NuGet.Config +++ b/src/Agent.Listener/NuGet.Config @@ -3,6 +3,7 @@ + diff --git a/src/Agent.Listener/SelfUpdater.cs b/src/Agent.Listener/SelfUpdater.cs index 31f5c231ee..716f24d33e 100644 --- a/src/Agent.Listener/SelfUpdater.cs +++ b/src/Agent.Listener/SelfUpdater.cs @@ -148,6 +148,12 @@ private async Task UpdateNeeded(string targetVersion, CancellationToken to Trace.Info($"Current running agent version is {BuildConstants.AgentPackage.Version}"); PackageVersion agentVersion = new PackageVersion(BuildConstants.AgentPackage.Version); + if (serverVersion.Major == 2 && agentVersion.Major == 3) + { + Trace.Info("We don't downgrade agent from 3.* to 2.*, skipping update"); + return false; + } + //Checking if current system support .NET 6 agent if (agentVersion.Major == 2 && serverVersion.Major == 3) { diff --git a/src/Agent.Plugins/TFCliManager.cs b/src/Agent.Plugins/TFCliManager.cs index b9a3fe096d..0b37a5c96c 100644 --- a/src/Agent.Plugins/TFCliManager.cs +++ b/src/Agent.Plugins/TFCliManager.cs @@ -47,7 +47,7 @@ public override TfsVCFeatures Features public async Task AddAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "add", localPath); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "add", localPath); } public void CleanupProxySetting() @@ -66,13 +66,13 @@ public Task EulaAsync() public async Task GetAsync(string localPath, bool quiet = false) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, quiet, RetriesOnFailure, "vc", "get", $"/version:{SourceVersion}", "/recursive", "/overwrite", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, quiet, RetriesOnFailure, "vc", "get", $"/version:{SourceVersion}", "/recursive", "/overwrite", localPath); } public string ResolvePath(string serverPath) { ArgUtil.NotNullOrEmpty(serverPath, nameof(serverPath)); - string localPath = RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "resolvePath", serverPath).GetAwaiter().GetResult(); + string localPath = RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "resolvePath", serverPath).GetAwaiter().GetResult(); return localPath?.Trim() ?? string.Empty; } @@ -84,7 +84,7 @@ public string ResolvePath(string serverPath) // // The current approach taken is: allow the exception to bubble. The TfsVCSourceProvider // will catch the exception, log it as a warning, throw away the workspace, and re-clone. - public async Task ScorchAsync() => await RunCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "scorch", SourcesDirectory, "/recursive", "/diff", "/unmapped"); + public async Task ScorchAsync() => await RunCommandAsync(FormatTags.OmitCollectionUrl, "vc", "scorch", SourcesDirectory, "/recursive", "/diff", "/unmapped"); public void SetupProxy(string proxyUrl, string proxyUsername, string proxyPassword) { @@ -173,11 +173,11 @@ public async Task ShelveAsync(string shelveset, string commentFile, bool move) // TODO: Remove parameter "move" after last-saved-checkin-metadata problem is fixed properly. if (move) { - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "shelve", "/move", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "shelve", "/move", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); return; } - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "shelve", "/saved", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "shelve", "/saved", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); } public async Task ShelvesetsAsync(string shelveset) @@ -241,13 +241,13 @@ public override async Task TryWorkspaceDeleteAsync(ITfsVCWorkspace workspa public async Task UndoAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "undo", "/recursive", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "vc", "undo", "/recursive", localPath); } public async Task UnshelveAsync(string shelveset, bool failOnNonZeroExitCode = true) { ArgUtil.NotNullOrEmpty(shelveset, nameof(shelveset)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, false, failOnNonZeroExitCode, "vc", "unshelve", shelveset); + await RunCommandAsync(FormatTags.OmitCollectionUrl, false, failOnNonZeroExitCode, "vc", "unshelve", shelveset); } public async Task WorkfoldCloakAsync(string serverPath) diff --git a/src/Agent.Plugins/TeeCliManager.cs b/src/Agent.Plugins/TeeCliManager.cs index 6daef88c87..4199a49430 100644 --- a/src/Agent.Plugins/TeeCliManager.cs +++ b/src/Agent.Plugins/TeeCliManager.cs @@ -28,7 +28,7 @@ public sealed class TeeCliManager : TfsVCCliManager, ITfsVCCliManager public async Task AddAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "add", localPath); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "add", localPath); } public void CleanupProxySetting() @@ -38,13 +38,13 @@ public void CleanupProxySetting() public async Task EulaAsync() { - await RunCommandAsync(FormatFlags.All, "eula", "-accept"); + await RunCommandAsync(FormatTags.All, "eula", "-accept"); } public async Task GetAsync(string localPath, bool quiet = false) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, quiet, 3, "get", $"-version:{SourceVersion}", "-recursive", "-overwrite", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, quiet, 3, "get", $"-version:{SourceVersion}", "-recursive", "-overwrite", localPath); } public string ResolvePath(string serverPath) @@ -135,11 +135,11 @@ public async Task ShelveAsync(string shelveset, string commentFile, bool move) // TODO: Remove parameter move after last-saved-checkin-metadata problem is fixed properly. if (move) { - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-move", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-move", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); return; } - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-saved", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-saved", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); } public async Task ShelvesetsAsync(string shelveset) @@ -166,7 +166,7 @@ public async Task ShelvesetsAsync(string shelveset) public async Task StatusAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - string output = await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "status", "-recursive", "-nodetect", "-format:xml", localPath); + string output = await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "status", "-recursive", "-nodetect", "-format:xml", localPath); string xml = ExtractXml(output); var serializer = new XmlSerializer(typeof(TeeStatus)); using (var reader = new StringReader(xml ?? string.Empty)) @@ -228,13 +228,13 @@ public override async Task TryWorkspaceDeleteAsync(ITfsVCWorkspace workspa public async Task UndoAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "undo", "-recursive", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "undo", "-recursive", localPath); } public async Task UnshelveAsync(string shelveset, bool failOnNonZeroExitCode = true) { ArgUtil.NotNullOrEmpty(shelveset, nameof(shelveset)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, false, failOnNonZeroExitCode, "unshelve", "-format:detailed", $"-workspace:{WorkspaceName}", shelveset); + await RunCommandAsync(FormatTags.OmitCollectionUrl, false, failOnNonZeroExitCode, "unshelve", "-format:detailed", $"-workspace:{WorkspaceName}", shelveset); } public async Task WorkfoldCloakAsync(string serverPath) @@ -280,7 +280,7 @@ public async Task WorkspacesAsync(bool matchWorkspaceNameOnAn args.Add("-format:xml"); // Run the command. - TfsVCPorcelainCommandResult result = await TryRunPorcelainCommandAsync(FormatFlags.None, RetriesOnFailure, args.ToArray()); + TfsVCPorcelainCommandResult result = await TryRunPorcelainCommandAsync(FormatTags.None, RetriesOnFailure, args.ToArray()); ArgUtil.NotNull(result, nameof(result)); if (result.Exception != null) { diff --git a/src/Agent.Plugins/TfsVCCliManager.cs b/src/Agent.Plugins/TfsVCCliManager.cs index f6a8646d2f..f9c7b41842 100644 --- a/src/Agent.Plugins/TfsVCCliManager.cs +++ b/src/Agent.Plugins/TfsVCCliManager.cs @@ -69,25 +69,25 @@ protected string WorkspaceName protected Task RunCommandAsync(params string[] args) { - return RunCommandAsync(FormatFlags.None, args); + return RunCommandAsync(FormatTags.None, args); } protected Task RunCommandAsync(int retriesOnFailure, params string[] args) { - return RunCommandAsync(FormatFlags.None, false, retriesOnFailure, args); + return RunCommandAsync(FormatTags.None, false, retriesOnFailure, args); } - protected Task RunCommandAsync(FormatFlags formatFlags, params string[] args) + protected Task RunCommandAsync(FormatTags formatFlags, params string[] args) { return RunCommandAsync(formatFlags, false, args); } - protected Task RunCommandAsync(FormatFlags formatFlags, bool quiet, params string[] args) + protected Task RunCommandAsync(FormatTags formatFlags, bool quiet, params string[] args) { return RunCommandAsync(formatFlags, quiet, 0, args); } - protected async Task RunCommandAsync(FormatFlags formatFlags, bool quiet, int retriesOnFailure, params string[] args) + protected async Task RunCommandAsync(FormatTags formatFlags, bool quiet, int retriesOnFailure, params string[] args) { for (int attempt = 0; attempt < retriesOnFailure - 1; attempt++) { @@ -119,7 +119,7 @@ private string WriteCommandToFile(string command) return temporaryName; } - protected async Task RunCommandAsync(FormatFlags formatFlags, bool quiet, bool failOnNonZeroExitCode, params string[] args) + protected async Task RunCommandAsync(FormatTags formatFlags, bool quiet, bool failOnNonZeroExitCode, params string[] args) { // Validation. ArgUtil.NotNull(args, nameof(args)); @@ -179,22 +179,22 @@ protected async Task RunCommandAsync(FormatFlags formatFlags, bool quiet, b } } - protected Task RunPorcelainCommandAsync(FormatFlags formatFlags, params string[] args) + protected Task RunPorcelainCommandAsync(FormatTags formatFlags, params string[] args) { return RunPorcelainCommandAsync(formatFlags, 0, args); } protected Task RunPorcelainCommandAsync(params string[] args) { - return RunPorcelainCommandAsync(FormatFlags.None, 0, args); + return RunPorcelainCommandAsync(FormatTags.None, 0, args); } protected Task RunPorcelainCommandAsync(int retriesOnFailure, params string[] args) { - return RunPorcelainCommandAsync(FormatFlags.None, retriesOnFailure, args); + return RunPorcelainCommandAsync(FormatTags.None, retriesOnFailure, args); } - protected async Task RunPorcelainCommandAsync(FormatFlags formatFlags, int retriesOnFailure, params string[] args) + protected async Task RunPorcelainCommandAsync(FormatTags formatFlags, int retriesOnFailure, params string[] args) { // Run the command. TfsVCPorcelainCommandResult result = await TryRunPorcelainCommandAsync(formatFlags, retriesOnFailure, args); @@ -211,7 +211,7 @@ protected async Task RunPorcelainCommandAsync(FormatFlags formatFlags, i return string.Join(Environment.NewLine, result.Output ?? new List()); } - protected async Task TryRunPorcelainCommandAsync(FormatFlags formatFlags, int retriesOnFailure, params string[] args) + protected async Task TryRunPorcelainCommandAsync(FormatTags formatFlags, int retriesOnFailure, params string[] args) { var result = await TryRunPorcelainCommandAsync(formatFlags, args); for (int attempt = 0; attempt < retriesOnFailure && result.Exception != null && result.Exception?.ExitCode != 1; attempt++) @@ -225,7 +225,7 @@ protected async Task TryRunPorcelainCommandAsync(Fo return result; } - protected async Task TryRunPorcelainCommandAsync(FormatFlags formatFlags, params string[] args) + protected async Task TryRunPorcelainCommandAsync(FormatTags formatFlags, params string[] args) { // Validation. ArgUtil.NotNull(args, nameof(args)); @@ -304,7 +304,7 @@ private void CleanupTfsVCOutput(ref TfsVCPorcelainCommandResult command, string command.Output.RemoveAll(item => stringsToRemove.Contains(item)); } - private string FormatArguments(FormatFlags formatFlags, params string[] args) + private string FormatArguments(FormatTags formatFlags, params string[] args) { // Validation. ArgUtil.NotNull(args, nameof(args)); @@ -331,7 +331,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } // Add the common parameters. - if (!formatFlags.HasFlag(FormatFlags.OmitCollectionUrl)) + if (!formatFlags.HasFlag(FormatTags.OmitCollectionUrl)) { if (Features.HasFlag(TfsVCFeatures.EscapedUrl)) { @@ -358,7 +358,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } } - if (!formatFlags.HasFlag(FormatFlags.OmitLogin)) + if (!formatFlags.HasFlag(FormatTags.OmitLogin)) { if (Features.HasFlag(TfsVCFeatures.LoginType)) { @@ -371,7 +371,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } } - if (!formatFlags.HasFlag(FormatFlags.OmitNoPrompt)) + if (!formatFlags.HasFlag(FormatTags.OmitNoPrompt)) { formattedArgs.Add($"{Switch}noprompt"); } @@ -380,7 +380,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } [Flags] - protected enum FormatFlags + protected enum FormatTags { None = 0, OmitCollectionUrl = 1, @@ -476,4 +476,4 @@ public interface ITfsVCMapping string ServerPath { get; } } -} \ No newline at end of file +} diff --git a/src/Agent.Sdk/Agent.Sdk.csproj b/src/Agent.Sdk/Agent.Sdk.csproj index 98c85e1c74..950d9bbcd5 100644 --- a/src/Agent.Sdk/Agent.Sdk.csproj +++ b/src/Agent.Sdk/Agent.Sdk.csproj @@ -10,6 +10,7 @@ + diff --git a/src/Agent.Sdk/Knob/AgentKnobs.cs b/src/Agent.Sdk/Knob/AgentKnobs.cs index 9f1eb168a3..950708d5ba 100644 --- a/src/Agent.Sdk/Knob/AgentKnobs.cs +++ b/src/Agent.Sdk/Knob/AgentKnobs.cs @@ -145,6 +145,13 @@ public class AgentKnobs new EnvironmentKnobSource("AGENT_USE_NODE10"), new BuiltInDefaultKnobSource("false")); + public static readonly Knob UseNode16 = new Knob( + nameof(UseNode16), + "Forces the agent to use Node 16 handler for all Node-based tasks", + new RuntimeKnobSource("AGENT_USE_NODE16"), + new EnvironmentKnobSource("AGENT_USE_NODE16"), + new BuiltInDefaultKnobSource("false")); + // Agent logging public static readonly Knob AgentPerflog = new Knob( nameof(AgentPerflog), diff --git a/src/Agent.Sdk/Util/AzureInstanceMetadataProvider.cs b/src/Agent.Sdk/Util/AzureInstanceMetadataProvider.cs index 94676ad490..1012878d7d 100644 --- a/src/Agent.Sdk/Util/AzureInstanceMetadataProvider.cs +++ b/src/Agent.Sdk/Util/AzureInstanceMetadataProvider.cs @@ -55,7 +55,7 @@ public string GetMetadata(string category, Dictionary parameters throw new ObjectDisposedException(nameof(AzureInstanceMetadataProvider)); } - HttpRequestMessage request = BuildRequest($"{_azureMetadataEndpoint}/{category}", parameters); + using HttpRequestMessage request = BuildRequest($"{_azureMetadataEndpoint}/{category}", parameters); HttpResponseMessage response = _client.SendAsync(request).Result; if (!response.IsSuccessStatusCode) diff --git a/src/Agent.Worker/Build/TFCommandManager.cs b/src/Agent.Worker/Build/TFCommandManager.cs index 54ceb2a2eb..d606ee16c4 100644 --- a/src/Agent.Worker/Build/TFCommandManager.cs +++ b/src/Agent.Worker/Build/TFCommandManager.cs @@ -44,7 +44,7 @@ public override TfsVCFeatures Features public async Task AddAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "add", localPath); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "add", localPath); } public void CleanupProxySetting() @@ -63,13 +63,13 @@ public Task EulaAsync() public async Task GetAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "get", $"/version:{SourceVersion}", "/recursive", "/overwrite", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "vc", "get", $"/version:{SourceVersion}", "/recursive", "/overwrite", localPath); } public string ResolvePath(string serverPath) { ArgUtil.NotNullOrEmpty(serverPath, nameof(serverPath)); - string localPath = RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "resolvePath", serverPath).GetAwaiter().GetResult(); + string localPath = RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "resolvePath", serverPath).GetAwaiter().GetResult(); return localPath?.Trim() ?? string.Empty; } @@ -81,7 +81,7 @@ public string ResolvePath(string serverPath) // // The current approach taken is: allow the exception to bubble. The TfsVCSourceProvider // will catch the exception, log it as a warning, throw away the workspace, and re-clone. - public async Task ScorchAsync() => await RunCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "scorch", SourcesDirectory, "/recursive", "/diff", "/unmapped"); + public async Task ScorchAsync() => await RunCommandAsync(FormatTags.OmitCollectionUrl, "vc", "scorch", SourcesDirectory, "/recursive", "/diff", "/unmapped"); public void SetupProxy(string proxyUrl, string proxyUsername, string proxyPassword) { @@ -170,11 +170,11 @@ public async Task ShelveAsync(string shelveset, string commentFile, bool move) // TODO: Remove parameter "move" after last-saved-checkin-metadata problem is fixed properly. if (move) { - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "shelve", "/move", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "shelve", "/move", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); return; } - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "shelve", "/saved", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "vc", "shelve", "/saved", "/replace", "/recursive", $"/comment:@{commentFile}", shelveset, SourcesDirectory); } public async Task ShelvesetsAsync(string shelveset) @@ -238,13 +238,13 @@ public async Task TryWorkspaceDeleteAsync(ITfsVCWorkspace workspace) public async Task UndoAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "undo", "/recursive", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "vc", "undo", "/recursive", localPath); } public async Task UnshelveAsync(string shelveset) { ArgUtil.NotNullOrEmpty(shelveset, nameof(shelveset)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "vc", "unshelve", shelveset); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "vc", "unshelve", shelveset); } public async Task WorkfoldCloakAsync(string serverPath) @@ -519,4 +519,4 @@ public sealed class TFMapping : ITfsVCMapping [XmlAttribute(AttributeName = "type", Namespace = "")] public string Type { get; set; } } -} \ No newline at end of file +} diff --git a/src/Agent.Worker/Build/TeeCommandManager.cs b/src/Agent.Worker/Build/TeeCommandManager.cs index 003364692a..3a816fda41 100644 --- a/src/Agent.Worker/Build/TeeCommandManager.cs +++ b/src/Agent.Worker/Build/TeeCommandManager.cs @@ -26,7 +26,7 @@ public sealed class TeeCommandManager : TfsVCCommandManager, ITfsVCCommandManage public async Task AddAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "add", localPath); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "add", localPath); } public void CleanupProxySetting() @@ -36,13 +36,13 @@ public void CleanupProxySetting() public async Task EulaAsync() { - await RunCommandAsync(FormatFlags.All, "eula", "-accept"); + await RunCommandAsync(FormatTags.All, "eula", "-accept"); } public async Task GetAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "get", $"-version:{SourceVersion}", "-recursive", "-overwrite", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "get", $"-version:{SourceVersion}", "-recursive", "-overwrite", localPath); } public string ResolvePath(string serverPath) @@ -131,11 +131,11 @@ public async Task ShelveAsync(string shelveset, string commentFile, bool move) // TODO: Remove parameter move after last-saved-checkin-metadata problem is fixed properly. if (move) { - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-move", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-move", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); return; } - await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-saved", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); + await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "shelve", $"-workspace:{WorkspaceName}", "-saved", "-replace", "-recursive", $"-comment:@{commentFile}", shelveset); } public async Task ShelvesetsAsync(string shelveset) @@ -162,7 +162,7 @@ public async Task ShelvesetsAsync(string shelveset) public async Task StatusAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - string output = await RunPorcelainCommandAsync(FormatFlags.OmitCollectionUrl, "status", "-recursive", "-nodetect", "-format:xml", localPath); + string output = await RunPorcelainCommandAsync(FormatTags.OmitCollectionUrl, "status", "-recursive", "-nodetect", "-format:xml", localPath); string xml = ExtractXml(output); var serializer = new XmlSerializer(typeof(TeeStatus)); using (var reader = new StringReader(xml ?? string.Empty)) @@ -226,7 +226,7 @@ public async Task TryWorkspaceDeleteAsync(ITfsVCWorkspace workspace) public async Task UndoAsync(string localPath) { ArgUtil.NotNullOrEmpty(localPath, nameof(localPath)); - await RunCommandAsync(FormatFlags.OmitCollectionUrl, "undo", "-recursive", localPath); + await RunCommandAsync(FormatTags.OmitCollectionUrl, "undo", "-recursive", localPath); } public async Task UnshelveAsync(string shelveset) @@ -278,7 +278,7 @@ public async Task WorkspacesAsync(bool matchWorkspaceNameOnAn args.Add("-format:xml"); // Run the command. - TfsVCPorcelainCommandResult result = await TryRunPorcelainCommandAsync(FormatFlags.None, false, args.ToArray()); + TfsVCPorcelainCommandResult result = await TryRunPorcelainCommandAsync(FormatTags.None, false, args.ToArray()); ArgUtil.NotNull(result, nameof(result)); if (result.Exception != null) { @@ -494,4 +494,4 @@ public sealed class TeeMapping : ITfsVCMapping [XmlAttribute(AttributeName = "server-item")] public string ServerPath { get; set; } } -} \ No newline at end of file +} diff --git a/src/Agent.Worker/Build/TfsVCCommandManager.cs b/src/Agent.Worker/Build/TfsVCCommandManager.cs index 81dea5bda3..6315832e48 100644 --- a/src/Agent.Worker/Build/TfsVCCommandManager.cs +++ b/src/Agent.Worker/Build/TfsVCCommandManager.cs @@ -106,10 +106,10 @@ protected string WorkspaceName protected Task RunCommandAsync(params string[] args) { - return RunCommandAsync(FormatFlags.None, args); + return RunCommandAsync(FormatTags.None, args); } - protected async Task RunCommandAsync(FormatFlags formatFlags, params string[] args) + protected async Task RunCommandAsync(FormatTags formatFlags, params string[] args) { // Validation. ArgUtil.NotNull(args, nameof(args)); @@ -161,20 +161,20 @@ await processInvoker.ExecuteAsync( protected Task RunPorcelainCommandAsync(params string[] args) { - return RunPorcelainCommandAsync(FormatFlags.None, args); + return RunPorcelainCommandAsync(FormatTags.None, args); } protected Task RunPorcelainCommandAsync(bool ignoreStderr, params string[] args) { - return RunPorcelainCommandAsync(FormatFlags.None, ignoreStderr, args); + return RunPorcelainCommandAsync(FormatTags.None, ignoreStderr, args); } - protected Task RunPorcelainCommandAsync(FormatFlags formatFlags, params string[] args) + protected Task RunPorcelainCommandAsync(FormatTags formatFlags, params string[] args) { return RunPorcelainCommandAsync(formatFlags, false, args); } - protected async Task RunPorcelainCommandAsync(FormatFlags formatFlags, bool ignoreStderr, params string[] args) + protected async Task RunPorcelainCommandAsync(FormatTags formatFlags, bool ignoreStderr, params string[] args) { // Run the command. TfsVCPorcelainCommandResult result = await TryRunPorcelainCommandAsync(formatFlags, ignoreStderr, args); @@ -200,7 +200,7 @@ private string WriteCommandToFile(string command) return temporaryName; } - protected async Task TryRunPorcelainCommandAsync(FormatFlags formatFlags, bool ignoreStderr, params string[] args) + protected async Task TryRunPorcelainCommandAsync(FormatTags formatFlags, bool ignoreStderr, params string[] args) { // Validation. ArgUtil.NotNull(args, nameof(args)); @@ -285,7 +285,7 @@ private void CleanupTfsVCOutput(ref TfsVCPorcelainCommandResult command, string command.Output.RemoveAll(item => stringsToRemove.Contains(item)); } - private string FormatArguments(FormatFlags formatFlags, params string[] args) + private string FormatArguments(FormatTags formatFlags, params string[] args) { // Validation. ArgUtil.NotNull(args, nameof(args)); @@ -312,7 +312,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } // Add the common parameters. - if (!formatFlags.HasFlag(FormatFlags.OmitCollectionUrl)) + if (!formatFlags.HasFlag(FormatTags.OmitCollectionUrl)) { if (Features.HasFlag(TfsVCFeatures.EscapedUrl)) { @@ -339,7 +339,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } } - if (!formatFlags.HasFlag(FormatFlags.OmitLogin)) + if (!formatFlags.HasFlag(FormatTags.OmitLogin)) { if (Features.HasFlag(TfsVCFeatures.LoginType)) { @@ -352,7 +352,7 @@ private string FormatArguments(FormatFlags formatFlags, params string[] args) } } - if (!formatFlags.HasFlag(FormatFlags.OmitNoPrompt)) + if (!formatFlags.HasFlag(FormatTags.OmitNoPrompt)) { formattedArgs.Add($"{Switch}noprompt"); } @@ -374,7 +374,7 @@ private string GetEndpointData(ServiceEndpoint endpoint, string name) } [Flags] - protected enum FormatFlags + protected enum FormatTags { None = 0, OmitCollectionUrl = 1, @@ -470,4 +470,4 @@ public interface ITfsVCMapping string ServerPath { get; } } -} \ No newline at end of file +} diff --git a/src/Agent.Worker/Handlers/NodeHandler.cs b/src/Agent.Worker/Handlers/NodeHandler.cs index 7b10842518..08584f33eb 100644 --- a/src/Agent.Worker/Handlers/NodeHandler.cs +++ b/src/Agent.Worker/Handlers/NodeHandler.cs @@ -11,6 +11,7 @@ using Newtonsoft.Json.Linq; using System.Text.RegularExpressions; using System.Linq; +using System.Threading; namespace Microsoft.VisualStudio.Services.Agent.Worker.Handlers { @@ -214,6 +215,7 @@ public async Task RunAsync() public string GetNodeLocation() { + bool useNode16 = AgentKnobs.UseNode16.GetValue(ExecutionContext).AsBoolean(); bool useNode10 = AgentKnobs.UseNode10.GetValue(ExecutionContext).AsBoolean(); bool taskHasNode10Data = Data is Node10HandlerData; bool taskHasNode16Data = Data is Node16HandlerData; @@ -236,6 +238,12 @@ public string GetNodeLocation() nodeFolder = NodeHandler.node10Folder; } + if (useNode16) + { + Trace.Info($"Found UseNode16 knob, using node16 for node tasks {useNode16}"); + nodeFolder = NodeHandler.node16Folder; + } + if (useNode10) { Trace.Info($"Found UseNode10 knob, use node10 for node tasks: {useNode10}"); diff --git a/src/Agent.Worker/Handlers/StepHost.cs b/src/Agent.Worker/Handlers/StepHost.cs index 7f9ca2cf99..aa0c2a07c7 100644 --- a/src/Agent.Worker/Handlers/StepHost.cs +++ b/src/Agent.Worker/Handlers/StepHost.cs @@ -247,4 +247,4 @@ private class ContainerStandardInPayload public string ExecutionHandlerPrependPath { get; set; } } } -} \ No newline at end of file +} diff --git a/src/Agent.Worker/JobRunner.cs b/src/Agent.Worker/JobRunner.cs index fbf9b1e647..2a36027981 100644 --- a/src/Agent.Worker/JobRunner.cs +++ b/src/Agent.Worker/JobRunner.cs @@ -40,6 +40,7 @@ public IJobServerQueue JobServerQueue { set => _jobServerQueue = value; } + public async Task RunAsync(Pipelines.AgentJobRequestMessage message, CancellationToken jobRequestCancellationToken) { // Validate parameters. @@ -83,9 +84,9 @@ public async Task RunAsync(Pipelines.AgentJobRequestMessage message, VssConnection jobConnection = VssUtil.CreateConnection( jobServerUrl, jobServerCredential, - trace: Trace, + Trace, new DelegatingHandler[] { new ThrottlingReportHandler(_jobServerQueue) } - ); + ); await jobServer.ConnectAsync(jobConnection); _jobServerQueue.Start(message); diff --git a/src/Agent.Worker/Variables.cs b/src/Agent.Worker/Variables.cs index 661248f3b2..6e851b72aa 100644 --- a/src/Agent.Worker/Variables.cs +++ b/src/Agent.Worker/Variables.cs @@ -398,7 +398,7 @@ public void Unset(string name) lock (_setLock) { Variable dummy; - _expanded.Remove(name, out dummy); + _expanded.Remove(name, out dummy); _nonexpanded.Remove(name, out dummy); _trace.Verbose($"Unset '{name}'"); } @@ -447,8 +447,7 @@ public void Set(string name, string val, bool secret = false, bool readOnly = fa public bool IsReadOnly(string name) { Variable existingVariable = null; - if (!_expanded.TryGetValue(name, out existingVariable)) - { + if (!_expanded.TryGetValue(name, out existingVariable)) { _nonexpanded.TryGetValue(name, out existingVariable); } diff --git a/src/Common.props b/src/Common.props index 1234681446..535fa1bddc 100644 --- a/src/Common.props +++ b/src/Common.props @@ -1,7 +1,7 @@ - netcoreapp3.1 - 3.1.32 + net6 + 6.0.13 $(PackageRuntime) true portable-net45+win8 @@ -12,6 +12,7 @@ 0.5.183-private $(CodeAnalysis) + false diff --git a/src/Microsoft.VisualStudio.Services.Agent/Microsoft.VisualStudio.Services.Agent.csproj b/src/Microsoft.VisualStudio.Services.Agent/Microsoft.VisualStudio.Services.Agent.csproj index 658f0f3311..30103259b4 100644 --- a/src/Microsoft.VisualStudio.Services.Agent/Microsoft.VisualStudio.Services.Agent.csproj +++ b/src/Microsoft.VisualStudio.Services.Agent/Microsoft.VisualStudio.Services.Agent.csproj @@ -14,6 +14,7 @@ + diff --git a/src/Microsoft.VisualStudio.Services.Agent/Util/VarUtil.cs b/src/Microsoft.VisualStudio.Services.Agent/Util/VarUtil.cs index 2206a681aa..0192a32b5c 100644 --- a/src/Microsoft.VisualStudio.Services.Agent/Util/VarUtil.cs +++ b/src/Microsoft.VisualStudio.Services.Agent/Util/VarUtil.cs @@ -12,7 +12,7 @@ namespace Microsoft.VisualStudio.Services.Agent.Util { - public static class VarUtil + public static class VarUtil { public static StringComparer EnvironmentVariableKeyComparer { diff --git a/src/Misc/dotnet-install.sh b/src/Misc/dotnet-install.sh old mode 100755 new mode 100644 diff --git a/src/Misc/externals.sh b/src/Misc/externals.sh old mode 100755 new mode 100644 index f3ca255a3e..36c6235bc8 --- a/src/Misc/externals.sh +++ b/src/Misc/externals.sh @@ -190,13 +190,22 @@ if [[ "$PACKAGERUNTIME" == "osx-x64" ]]; then acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/node-v${NODE16_VERSION}-darwin-x64.tar.gz" node16 fix_nested_dir fi + +if [[ "$PACKAGERUNTIME" == "osx-arm64" ]]; then + if [[ "$INCLUDE_NODE6" == "true" ]]; then + acquireExternalTool "$NODE_URL/v${NODE_VERSION}/node-v${NODE_VERSION}-darwin-x64.tar.gz" node fix_nested_dir + fi + acquireExternalTool "$NODE_URL/v${NODE10_VERSION}/node-v${NODE10_VERSION}-darwin-x64.tar.gz" node10 fix_nested_dir + acquireExternalTool "$NODE_URL/v${NODE16_VERSION}/node-v${NODE16_VERSION}-darwin-arm64.tar.gz" node16 fix_nested_dir +fi + # Download the external tools common across OSX and Linux PACKAGERUNTIMEs. -if [[ "$PACKAGERUNTIME" == "linux-x64" || "$PACKAGERUNTIME" == "linux-arm" || "$PACKAGERUNTIME" == "linux-arm64" || "$PACKAGERUNTIME" == "osx-x64" || "$PACKAGERUNTIME" == "rhel.6-x64" ]]; then +if [[ "$PACKAGERUNTIME" == "linux-x64" || "$PACKAGERUNTIME" == "linux-arm" || "$PACKAGERUNTIME" == "linux-arm64" || "$PACKAGERUNTIME" == "osx-x64" || "$PACKAGERUNTIME" == "osx-arm64" || "$PACKAGERUNTIME" == "rhel.7.2-x64" ]]; then acquireExternalTool "$CONTAINER_URL/vso-task-lib/0.5.5/vso-task-lib.tar.gz" vso-task-lib fi # Download the external tools common across Linux PACKAGERUNTIMEs (excluding OSX). -if [[ "$PACKAGERUNTIME" == "linux-x64" || "$PACKAGERUNTIME" == "rhel.6-x64" ]]; then +if [[ "$PACKAGERUNTIME" == "linux-x64" || "$PACKAGERUNTIME" == "rhel.7.2-x64" ]]; then if [[ "$INCLUDE_NODE6" == "true" ]]; then acquireExternalTool "$NODE_URL/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.gz" node fix_nested_dir fi @@ -237,6 +246,13 @@ if [[ "$PACKAGERUNTIME" != "win-x64" && "$PACKAGERUNTIME" != "win-x86" ]]; then rm "$LAYOUT_DIR/externals/node16/bin/corepack" fi +if [[ "$PACKAGERUNTIME" != "win-x64" && "$PACKAGERUNTIME" != "win-x86" ]]; then + rm -rf "$LAYOUT_DIR/externals/node/lib/node_modules/npm" + rm "$LAYOUT_DIR/externals/node/bin/npm" + rm -rf "$LAYOUT_DIR/externals/node10/lib/node_modules/npm" + rm "$LAYOUT_DIR/externals/node10/bin/npm" +fi + if [[ "$L1_MODE" != "" || "$PRECACHE" != "" ]]; then # cmdline task acquireExternalTool "$CONTAINER_URL/l1Tasks/d9bafed4-0b18-4f58-968d-86655b4d2ce9.zip" "Tasks" false dont_uncompress diff --git a/src/Misc/layoutbin/installdependencies.sh b/src/Misc/layoutbin/installdependencies.sh old mode 100755 new mode 100644 index f67681e558..1f96454b45 --- a/src/Misc/layoutbin/installdependencies.sh +++ b/src/Misc/layoutbin/installdependencies.sh @@ -46,6 +46,12 @@ function print_rhel6errormessage() echo "https://github.com/dotnet/core/blob/main/Documentation/build-and-install-rhel6-prerequisites.md" } +function print_rhel6depricationmessage() +{ + echo "Detected Operation System is not supported by .NET 6 which is required to run this software" + echo "You can check supported OS on the following documentation: https://github.com/dotnet/core/blob/main/release-notes/6.0/supported-os.md" +} + if [ -e /etc/os-release ] then echo "--------OS Information--------" @@ -291,31 +297,12 @@ then fi elif [ -e /etc/redhat-release ] # RHEL6 doesn't have an os-release file defined, read redhat-release instead +# We no longer support RHEL6 then redhatRelease=$( /dev/null if [ $? -ne 0 ] then @@ -18,34 +32,27 @@ then exit 1 fi - ldd ./bin/libcoreclr.so | grep 'not found' + ldd ./bin/libcoreclr.so | grep -E 'not found|No such' if [ $? -eq 0 ]; then - echo "Dependencies is missing for .NET Core 3.1" + echo "Dependencies is missing for .NET Core 6.0" echo "Execute ./bin/installdependencies.sh to install any missing dependencies." exit 1 fi - ldd ./bin/System.Security.Cryptography.Native.OpenSsl.so | grep 'not found' + ldd ./bin/libSystem.Security.Cryptography.Native.OpenSsl.so | grep -E 'not found|No such' if [ $? -eq 0 ]; then - echo "Dependencies missing for .NET Core 3.1" + echo "Dependencies missing for .NET 6.0" echo "Execute ./bin/installdependencies.sh to install any missing dependencies." exit 1 fi - ldd ./bin/System.IO.Compression.Native.so | grep 'not found' + ldd ./bin/libSystem.IO.Compression.Native.so | grep -E 'not found|No such' if [ $? -eq 0 ]; then - echo "Dependencies missing for .NET Core 3.1" + echo "Dependencies missing for .NET 6.0" echo "Execute ./bin/installdependencies.sh to install any missing dependencies." exit 1 fi - ldd ./bin/System.Net.Http.Native.so | grep 'not found' - if [ $? -eq 0 ]; then - echo "Warning: on some platforms, libcurl3 is required." - echo "It was not found." - echo "Execute ./bin/installdependencies.sh to install missing dependencies." - fi - if ! [ -x "$(command -v ldconfig)" ]; then LDCONFIG_COMMAND="/sbin/ldconfig" if ! [ -x "$LDCONFIG_COMMAND" ]; then @@ -59,7 +66,7 @@ then libpath=${LD_LIBRARY_PATH:-} $LDCONFIG_COMMAND -NXv ${libpath//:/} 2>&1 | grep libicu >/dev/null 2>&1 if [ $? -ne 0 ]; then - echo "libicu's dependencies missing for .NET Core 3.1" + echo "libicu's dependencies missing for .NET 6" echo "Execute ./bin/installdependencies.sh to install any missing dependencies." exit 1 fi diff --git a/src/Misc/layoutroot/env.sh b/src/Misc/layoutroot/env.sh index 0bf710038e..6710a5a328 100755 --- a/src/Misc/layoutroot/env.sh +++ b/src/Misc/layoutroot/env.sh @@ -1,13 +1,13 @@ #!/bin/bash varCheckList=( - 'LANG' - 'JAVA_HOME' - 'ANT_HOME' - 'M2_HOME' - 'ANDROID_HOME' - 'GRADLE_HOME' - 'NVM_BIN' + 'LANG' + 'JAVA_HOME' + 'ANT_HOME' + 'M2_HOME' + 'ANDROID_HOME' + 'GRADLE_HOME' + 'NVM_BIN' 'NVM_PATH' 'VSTS_HTTP_PROXY' 'VSTS_HTTP_PROXY_USERNAME' @@ -34,7 +34,7 @@ function writeVar() if [ ! -z "${!checkVar}" ]; then echo "${checkVar}=${!checkVar}">>.env fi - fi + fi } echo $PATH>.path diff --git a/src/Test/L0/ConstantGenerationL0.cs b/src/Test/L0/ConstantGenerationL0.cs index 4e3dfb2d26..65f2cc26f9 100644 --- a/src/Test/L0/ConstantGenerationL0.cs +++ b/src/Test/L0/ConstantGenerationL0.cs @@ -21,7 +21,9 @@ public void BuildConstantGenerateSucceed() "linux-arm", "linux-arm64", "rhel.6-x64", - "osx-x64" + "rhel.7.2-x64", + "osx-x64", + "osx-arm64" }; Assert.True(BuildConstants.Source.CommitHash.Length == 40, $"CommitHash should be SHA-1 hash {BuildConstants.Source.CommitHash}"); diff --git a/src/Test/L0/HostContextExtensionL0.cs b/src/Test/L0/HostContextExtensionL0.cs index 45554a8bd7..3ed7dee1d5 100644 --- a/src/Test/L0/HostContextExtensionL0.cs +++ b/src/Test/L0/HostContextExtensionL0.cs @@ -15,6 +15,7 @@ public sealed class HostContextExtensionL0 [Fact] [Trait("Level", "L0")] [Trait("Category", "Common")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public void CreateHttpClientHandlerForCertValidationSkipCert() { // Arrange. @@ -31,6 +32,7 @@ public void CreateHttpClientHandlerForCertValidationSkipCert() [Fact] [Trait("Level", "L0")] [Trait("Category", "Common")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public void CreateHttpClientHandlerForCertValidationDontSkipCert() { // Arrange. diff --git a/src/Test/L0/Listener/AgentL0.cs b/src/Test/L0/Listener/AgentL0.cs index f63edb52e9..5ec33cc5c5 100644 --- a/src/Test/L0/Listener/AgentL0.cs +++ b/src/Test/L0/Listener/AgentL0.cs @@ -66,6 +66,7 @@ private JobCancelMessage CreateJobCancelMessage() [Trait("Level", "L0")] [Trait("Category", "Agent")] //process 2 new job messages, and one cancel message + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void TestRunAsync() { using (var hc = new TestHostContext(this)) @@ -703,6 +704,7 @@ public async void TestConfigureCLI(string arg, bool IsConfigured, int expected = [Trait("Level", "L0")] [Trait("Category", "Agent")] //process 1 job message and one metadata message + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void TestMetadataUpdate() { using (var hc = new TestHostContext(this)) diff --git a/src/Test/L0/Listener/Configuration/AgentCapabilitiesProviderTestL0.cs b/src/Test/L0/Listener/Configuration/AgentCapabilitiesProviderTestL0.cs index fb8929b8cd..c5e5e9e902 100644 --- a/src/Test/L0/Listener/Configuration/AgentCapabilitiesProviderTestL0.cs +++ b/src/Test/L0/Listener/Configuration/AgentCapabilitiesProviderTestL0.cs @@ -51,17 +51,17 @@ public async void TestInteractiveSessionCapability() using (var tokenSource = new CancellationTokenSource()) { hc.StartupType = StartupType.AutoStartup; - await VerifyInteractiveSessionCapability(hc, tokenSource.Token, true); + await VerifyInteractiveSessionCapability(hc, true, tokenSource.Token); hc.StartupType = StartupType.Service; - await VerifyInteractiveSessionCapability(hc, tokenSource.Token, false); + await VerifyInteractiveSessionCapability(hc, false, tokenSource.Token); hc.StartupType = StartupType.Manual; - await VerifyInteractiveSessionCapability(hc, tokenSource.Token, true); + await VerifyInteractiveSessionCapability(hc, true, tokenSource.Token); } } - private async Task VerifyInteractiveSessionCapability(IHostContext hc, CancellationToken token, bool expectedValue) + private async Task VerifyInteractiveSessionCapability(IHostContext hc, bool expectedValue, CancellationToken token) { // Arrange var provider = new AgentCapabilitiesProvider(); diff --git a/src/Test/L0/Listener/JobDispatcherL0.cs b/src/Test/L0/Listener/JobDispatcherL0.cs index da389c2c80..3195a54975 100644 --- a/src/Test/L0/Listener/JobDispatcherL0.cs +++ b/src/Test/L0/Listener/JobDispatcherL0.cs @@ -93,6 +93,7 @@ public async void DispatchesJobRequest() [Fact] [Trait("Level", "L0")] [Trait("Category", "Agent")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void DispatcherRenewJobRequest() { //Arrange @@ -151,6 +152,7 @@ public async void DispatcherRenewJobRequest() [Fact] [Trait("Level", "L0")] [Trait("Category", "Agent")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void DispatcherRenewJobRequestStopOnJobNotFoundExceptions() { //Arrange @@ -210,6 +212,7 @@ public async void DispatcherRenewJobRequestStopOnJobNotFoundExceptions() [Fact] [Trait("Level", "L0")] [Trait("Category", "Agent")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions() { //Arrange @@ -269,6 +272,7 @@ public async void DispatcherRenewJobRequestStopOnJobTokenExpiredExceptions() [Fact] [Trait("Level", "L0")] [Trait("Category", "Agent")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void DispatcherRenewJobRequestRecoverFromExceptions() { //Arrange @@ -330,6 +334,7 @@ public async void DispatcherRenewJobRequestRecoverFromExceptions() [Fact] [Trait("Level", "L0")] [Trait("Category", "Agent")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void DispatcherRenewJobRequestFirstRenewRetrySixTimes() { //Arrange @@ -385,6 +390,7 @@ public async void DispatcherRenewJobRequestFirstRenewRetrySixTimes() [Fact] [Trait("Level", "L0")] [Trait("Category", "Agent")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public async void DispatcherRenewJobRequestStopOnExpiredRequest() { //Arrange diff --git a/src/Test/L0/Plugin/ChunkerTests.cs b/src/Test/L0/Plugin/ChunkerTests.cs index 9ba288bfd8..a015e54acc 100644 --- a/src/Test/L0/Plugin/ChunkerTests.cs +++ b/src/Test/L0/Plugin/ChunkerTests.cs @@ -37,6 +37,7 @@ public class ChunkerTests // } // } + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA5394: Do not use insecure randomness")] private static void FillBufferWithTestContent(int seed, byte[] bytes) { var r = new Random(seed); diff --git a/src/Test/L0/Plugin/FingerprintCreatorTests.cs b/src/Test/L0/Plugin/FingerprintCreatorTests.cs index b2a091ded7..5c8fb2ead1 100644 --- a/src/Test/L0/Plugin/FingerprintCreatorTests.cs +++ b/src/Test/L0/Plugin/FingerprintCreatorTests.cs @@ -26,6 +26,8 @@ public class FingerprintCreatorTests private static readonly string path1; private static readonly string path2; + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA5394:Do not use insecure randomness")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Performance", "CA1810:Initialize all static fields")] static FingerprintCreatorTests() { var r = new Random(0); diff --git a/src/Test/L0/Plugin/LogPluginHostL0.cs b/src/Test/L0/Plugin/LogPluginHostL0.cs index d8e330b3d7..83de4da126 100644 --- a/src/Test/L0/Plugin/LogPluginHostL0.cs +++ b/src/Test/L0/Plugin/LogPluginHostL0.cs @@ -159,7 +159,7 @@ public Task ProcessLineAsync(IAgentLogPluginContext context, Pipelines.TaskStepD } } - public class TestPluginException : IAgentLogPlugin + public class PluginExceptionTest : IAgentLogPlugin { public string FriendlyName => "TestException"; @@ -413,7 +413,7 @@ public async Task LogPluginHost_HandleInitialExceptions() AgentLogPluginHostContext hostContext = CreateTestLogPluginHostContext(); hostContext.Variables["throw_initialize"] = "1"; - List plugins = new List() { new TestPlugin1(), new TestPluginException() }; + List plugins = new List() { new TestPlugin1(), new PluginExceptionTest() }; TestTrace trace = new TestTrace(tc); AgentLogPluginHost logPluginHost = new AgentLogPluginHost(hostContext, plugins, trace); @@ -447,7 +447,7 @@ public async Task LogPluginHost_HandleProcessExceptions() AgentLogPluginHostContext hostContext = CreateTestLogPluginHostContext(); hostContext.Variables["throw_process"] = "1"; - List plugins = new List() { new TestPlugin1(), new TestPluginException() }; + List plugins = new List() { new TestPlugin1(), new PluginExceptionTest() }; TestTrace trace = new TestTrace(tc); AgentLogPluginHost logPluginHost = new AgentLogPluginHost(hostContext, plugins, trace); diff --git a/src/Test/L0/Plugin/TestFileShareProvider/TestFileShareProviderL0.cs b/src/Test/L0/Plugin/TestFileShareProvider/TestFileShareProviderL0.cs index a2d834b6d6..3798b3db27 100644 --- a/src/Test/L0/Plugin/TestFileShareProvider/TestFileShareProviderL0.cs +++ b/src/Test/L0/Plugin/TestFileShareProvider/TestFileShareProviderL0.cs @@ -163,6 +163,7 @@ private void TestCleanup() } } + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA5394:Do not use insecure randomness")] private byte[] GenerateRandomData() { byte[] data = new byte[1024]; @@ -173,9 +174,9 @@ private byte[] GenerateRandomData() private byte[] ComputeHash(string filePath) { - using (var md5 = MD5.Create()) + using (SHA256 sha256Hash = SHA256.Create()) { - return md5.ComputeHash(File.ReadAllBytes(filePath)); + return sha256Hash.ComputeHash(File.ReadAllBytes(filePath)); } } } diff --git a/src/Test/L0/Plugin/TestResultParser/EnumerableExtensionL0.cs b/src/Test/L0/Plugin/TestResultParser/EnumerableExtensionL0.cs index de6b045943..80445514fe 100644 --- a/src/Test/L0/Plugin/TestResultParser/EnumerableExtensionL0.cs +++ b/src/Test/L0/Plugin/TestResultParser/EnumerableExtensionL0.cs @@ -13,6 +13,7 @@ public class EnumerableExtensionL0 [Fact] [Trait("Level", "L0")] [Trait("Category", "Plugin")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA5394:Do not use insecure randomness")] public void ListShouldBeBatchedAsPerRequestedSize() { const int listSize = 2500; @@ -29,6 +30,7 @@ public void ListShouldBeBatchedAsPerRequestedSize() [Fact] [Trait("Level", "L0")] [Trait("Category", "Plugin")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA5394: Do not use insecure randomness")] public void ListShouldBeBatchedIfSizeIsLessThanBatch() { const int listSize = 50; diff --git a/src/Test/L0/ProcessExtensionL0.cs b/src/Test/L0/ProcessExtensionL0.cs index 94ca411685..732ae9092e 100644 --- a/src/Test/L0/ProcessExtensionL0.cs +++ b/src/Test/L0/ProcessExtensionL0.cs @@ -17,6 +17,9 @@ public sealed class ProcessExtensionL0 [Trait("SkipOn", "darwin")] public async Task SuccessReadProcessEnv() { + // With the latest update coming with macOS 10.5.7+ there are no more any ways to retrieve + // information about the environment on the running process with ps utility. + // This test should be updated accordingly, disabling it while issue is under investigation using (TestHostContext hc = new TestHostContext(this)) { Tracing trace = hc.GetTrace(); diff --git a/src/Test/L0/TestHostContext.cs b/src/Test/L0/TestHostContext.cs index e548bf9a7a..6310acc404 100644 --- a/src/Test/L0/TestHostContext.cs +++ b/src/Test/L0/TestHostContext.cs @@ -38,6 +38,8 @@ public sealed class TestHostContext : IHostContext, IDisposable public CancellationToken AgentShutdownToken => _agentShutdownTokenSource.Token; public ShutdownReason AgentShutdownReason { get; private set; } public ILoggedSecretMasker SecretMasker => _secretMasker; + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Maintainability", "CA2000:Dispose objects before losing scope")] public TestHostContext(object testClass, [CallerMemberName] string testName = "") { ArgUtil.NotNull(testClass, nameof(testClass)); diff --git a/src/Test/L0/Util/IOUtilL0.cs b/src/Test/L0/Util/IOUtilL0.cs index 536eb2a64f..54e7813d78 100644 --- a/src/Test/L0/Util/IOUtilL0.cs +++ b/src/Test/L0/Util/IOUtilL0.cs @@ -986,6 +986,7 @@ await processInvoker.ExecuteAsync( [Fact] [Trait("Level", "L0")] [Trait("Category", "Common")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Performance", "CA1814:Prefer jagged arrays over multidimensional")] public void GetDirectoryName_LinuxStyle() { using (TestHostContext hc = new TestHostContext(this)) @@ -1009,6 +1010,7 @@ public void GetDirectoryName_LinuxStyle() [Fact] [Trait("Level", "L0")] [Trait("Category", "Common")] + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Performance", "CA1814:Prefer jagged arrays over multidimensional")] public void GetDirectoryName_WindowsStyle() { using (TestHostContext hc = new TestHostContext(this)) diff --git a/src/Test/L1/Mock/FakeConfigurationStore.cs b/src/Test/L1/Mock/FakeConfigurationStore.cs index 04762dac85..8699f321a6 100644 --- a/src/Test/L1/Mock/FakeConfigurationStore.cs +++ b/src/Test/L1/Mock/FakeConfigurationStore.cs @@ -13,7 +13,7 @@ public class FakeConfigurationStore : AgentService, IConfigurationStore public string RootFolder => Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + "/TestRuns/" + WorkingDirectoryName; - public List SetupInfo => new List(); + public List setupInfo => new List(); private AgentSettings _agentSettings; @@ -44,7 +44,7 @@ public CredentialData GetCredentials() public IEnumerable GetSetupInfo() { - return SetupInfo; + return setupInfo; } public AgentSettings GetSettings() diff --git a/src/agentversion b/src/agentversion index 871cd719de..e6501e63d0 100644 --- a/src/agentversion +++ b/src/agentversion @@ -1 +1 @@ -2.999.999 +2.999.999 \ No newline at end of file diff --git a/src/dev.sh b/src/dev.sh index 9401b0e937..639c7492b6 100755 --- a/src/dev.sh +++ b/src/dev.sh @@ -20,7 +20,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" source "$SCRIPT_DIR/.helpers.sh" DOTNETSDK_ROOT="$SCRIPT_DIR/../_dotnetsdk" -DOTNETSDK_VERSION="3.1.426" +DOTNETSDK_VERSION="6.0.405" DOTNETSDK_INSTALLDIR="$DOTNETSDK_ROOT/$DOTNETSDK_VERSION" AGENT_VERSION=$(cat "$SCRIPT_DIR/agentversion" | head -n 1 | tr -d "\n\r") @@ -68,10 +68,19 @@ function detect_platform_and_runtime_id () if [[ $redhatRelease == "CentOS release 6."* || $redhatRelease == "Red Hat Enterprise Linux Server release 6."* ]]; then DETECTED_RUNTIME_ID='rhel.6-x64' fi + if [[ $redhatRelease == "CentOS release 7."* || $redhatRelease == "Red Hat Enterprise Linux Server release 7."* ]]; then + DETECTED_RUNTIME_ID='rhel.7.2-x64' + fi fi elif [[ "$CURRENT_PLATFORM" == 'darwin' ]]; then DETECTED_RUNTIME_ID='osx-x64' + if command -v uname > /dev/null; then + local CPU_NAME=$(uname -m) + case $CPU_NAME in + arm64) DETECTED_RUNTIME_ID="osx-arm64";; + esac + fi fi } @@ -285,7 +294,7 @@ else RUNTIME_ID=$DETECTED_RUNTIME_ID fi -_VALID_RIDS='linux-x64:linux-arm:linux-arm64:rhel.6-x64:osx-x64:win-x64:win-x86' +_VALID_RIDS='linux-x64:linux-arm:linux-arm64:rhel.6-x64:rhel.7.2-x64:osx-x64:win-x64:win-x86:osx-arm64' if [[ ":$_VALID_RIDS:" != *:$RUNTIME_ID:* ]]; then failed "must specify a valid target runtime ID (one of: $_VALID_RIDS)" fi diff --git a/tools/CredScanRegexes/readme.md b/tools/CredScanRegexes/readme.md index de2a84c448..2aacc486e7 100644 --- a/tools/CredScanRegexes/readme.md +++ b/tools/CredScanRegexes/readme.md @@ -9,7 +9,7 @@ We have secured permission to ship a small subset of CredScan -- its set of batt This tool extracts the regexes from CredScan's knowledge base and outputs them in a format suitable for shipping with the agent. Requirements: -- .NET Core 3.1 or higher +- .NET 6.0 or higher - The [Azure Artifacts credential provider](https://github.com/Microsoft/artifacts-credprovider) - Access to the "msazure" org's [CredScanSDK feed](https://msazure.visualstudio.com/One/_packaging?_a=feed&feed=CredScanSDK)