.azurepipelines: Add Platform CI template

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570

Add template for Platform CI steps for a Pytools based build.
Add README to describe the template and how to use it.
Add helpful information for working with azurepipelines, templates, and
lessons learned.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>

Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
This commit is contained in:
Sean Brogan
2020-04-07 21:35:30 -07:00
committed by mergify[bot]
parent 0f1946b662
commit 4fcfd089aa
3 changed files with 243 additions and 0 deletions

View File

@@ -0,0 +1,59 @@
# CI Templates
This folder contains azure pipeline yml templates for "Core" and "Platform" Continuous Integration and PR validation.
## Common CI templates
### basetools-build-steps.yml
This template compiles the Edk2 basetools from source. The steps in this template are
conditional and will only run if variable `pkg_count` is greater than 0.
It also has two conditional steps only used when the toolchain contains GCC. These two steps
use `apt` to update the system packages and add those necessary for Edk2 builds.
## Core CI templates
### pr-gate-build-job.yml
This templates contains the jobs and most importantly the matrix of which packages and
targets to run for Core CI.
### pr-gate-steps.yml
This template is the main Core CI template. It controls all the steps run and is responsible for most functionality of the Core CI process. This template sets
the `pkg_count` variable using the `stuart_pr_eval` tool when the
build type is "pull request"
### spell-check-prereq-steps.yml
This template installs the node based tools used by the spell checker plugin. The steps
in this template are conditional and will only run if variable `pkg_count` is greater than 0.
## Platform CI templates
### platform-build-run-steps.yml
This template makes heavy use of pytools to build and run a platform in the Edk2 repo
Also uses basetools-build-steps.yml to compile basetools
#### Special Notes
* For a build type of pull request it will conditionally build if the patches change files that impact the platform.
* uses `stuart_pr_eval` to determine impact
* For manual builds or CI builds it will always build the platform
* It compiles basetools from source
* Will use `stuart_build --FlashOnly` to attempt to run the built image if the `Run` parameter is set.
* See the parameters block for expected configuration options
* Parameter `extra_install_step` allows the caller to insert extra steps. This is useful if additional dependencies, tools, or other things need to be installed. Here is an example of installing qemu on Windows.
``` yaml
steps:
- template: ../../.azurepipelines/templates/build-run-steps.yml
parameters:
extra_install_step:
- powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
condition: and(gt(variables.pkg_count, 0), succeeded())
```

View File

@@ -0,0 +1,134 @@
## @file
# File steps.yml
#
# template file containing the steps to build
#
# Copyright (c) Microsoft Corporation.
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
parameters:
- name: tool_chain_tag
type: string
default: ''
- name: build_pkg
type: string
default: ''
- name: build_target
type: string
default: ''
- name: build_arch
type: string
default: ''
- name: build_file
type: string
default: ''
- name: build_flags
type: string
default: ''
- name: run_flags
type: string
default: ''
- name: extra_install_step
type: stepList
default: []
steps:
- checkout: self
clean: true
fetchDepth: 1
- task: UsePythonVersion@0
inputs:
versionSpec: "3.8.x"
architecture: "x64"
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
# Set default
- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
# trim the package list if this is a PR
- task: CmdLine@1
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
condition: eq(variables['Build.Reason'], 'PullRequest')
# Setup repo
- task: CmdLine@1
displayName: Setup
inputs:
filename: stuart_setup
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
condition: and(gt(variables.pkg_count, 0), succeeded())
# Stuart Update
- task: CmdLine@1
displayName: Update
inputs:
filename: stuart_update
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
condition: and(gt(variables.pkg_count, 0), succeeded())
# build basetools
# do this after setup and update so that code base dependencies
# are all resolved.
- template: basetools-build-steps.yml
parameters:
tool_chain_tag: ${{ parameters.tool_chain_tag }}
# Potential Extra steps
- ${{ parameters.extra_install_step }}
# Build
- task: CmdLine@1
displayName: Build
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
condition: and(gt(variables.pkg_count, 0), succeeded())
# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1
# Copy the build logs to the artifact staging directory
- task: CopyFiles@2
displayName: "Copy build logs"
inputs:
targetFolder: "$(Build.ArtifactStagingDirectory)"
SourceFolder: "Build"
contents: |
BUILDLOG_*.txt
BUILDLOG_*.md
CI_*.txt
CI_*.md
CISETUP.txt
SETUPLOG.txt
UPDATE_LOG.txt
PREVALLOG.txt
TestSuites.xml
**/BUILD_TOOLS_REPORT.html
**/OVERRIDELOG.TXT
BASETOOLS_BUILD*.*
flattenFolders: true
condition: succeededOrFailed()
# Publish build artifacts to Azure Artifacts/TFS or a file share
- task: PublishBuildArtifacts@1
continueOnError: true
displayName: "Publish build logs"
inputs:
pathtoPublish: "$(Build.ArtifactStagingDirectory)"
artifactName: "Build Logs $(System.JobName)"
condition: succeededOrFailed()