BaseTools: Fix the issue caused by tostring() removal on Py39

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3136

Python 3.9 remove the array.array.tostring and
array.array.fromstring() function. This patch
is to use other method to replace tostring() and
fromstring()

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Mingyue Liang <mingyuex.liang@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

.azurepipelines/ReadMe.md

@@ -0,0 +1,50 @@
+# Azure DevOps Pipelines
+
+These yml files are used to provide CI builds using the Azure DevOps Pipeline Service.
+Most of the CI leverages edk2-pytools to support cross platform building and execution.
+
+## Core CI
+
+Focused on building and testing all packages in Edk2 without an actual target platform.
+
+See `.pytools/ReadMe.py` for more details
+
+## Platform CI
+
+Focused on building a single target platform and confirming functionality on that platform.
+
+## Conventions
+
+* Files extension should be *.yml.  *.yaml is also supported but in Edk2 we use those for our package configuration.
+* Platform CI files should be in the `<PlatformPkg>/.azurepipelines` folder.
+* Core CI files are in the root folder.
+* Shared templates are in the `templates` folder.
+* Top level CI files should be named `<host os>-<tool_chain_tag>.yml`
+
+## Links
+
+* Basic Azure Landing Site - https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops
+* Pipeline jobs - https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
+* Pipeline yml scheme - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
+* Pipeline expression - https://docs.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
+* PyTools - https://github.com/tianocore/edk2-pytool-extensions and https://github.com/tianocore/edk2-pytool-library
+
+## Lessons Learned
+
+### Templates and parameters
+
+They are great but evil.  If they are used as part of determining the steps of a build they must resolve before the build starts.  They can not use variables set in a yml or determined as part of a matrix.  If they are used in a step then they can be bound late.
+
+### File matching patterns
+
+On Linux this can hang if there are too many files in the search list.
+
+### Templates and file splitting
+
+Suggestion is to do one big yaml file that does what you want for one of your targets.  Then do the second one and find the deltas.  From that you can start to figure out the right split of files, steps, jobs.
+
+### Conditional steps
+
+If you want the step to show up in the log but not run, use a step conditional. This is great when a platform doesn't currently support a feature but you want the builders to know that the features exists and maybe someday it will.
+
+If you want the step to not show up use a template step conditional wrapper.  Beware this will be evaluated early (at build start).  This can hide things not needed on a given OS for example.

.azurepipelines/Ubuntu-GCC5.yml

@@ -2,6 +2,7 @@
 # Azure Pipeline build file for a build using ubuntu and GCC5
 #
 # Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 ##
 trigger:
@@ -14,5 +15,5 @@ jobs:
   parameters:
     tool_chain_tag: 'GCC5'
     vm_image: 'ubuntu-latest'
-    arch_list: "IA32,X64,ARM,AARCH64"
+    arch_list: "IA32,X64,ARM,AARCH64,RISCV64"
 

.azurepipelines/templates/ReadMe.md

@@ -0,0 +1,59 @@
+# CI Templates
+
+This folder contains azure pipeline yml templates for "Core" and "Platform" Continuous Integration and PR validation.
+
+## Common CI templates
+
+### basetools-build-steps.yml
+
+This template compiles the Edk2 basetools from source.  The steps in this template are
+conditional and will only run if variable `pkg_count` is greater than 0.
+
+It also has two conditional steps only used when the toolchain contains GCC. These two steps
+use `apt` to update the system packages and add those necessary for Edk2 builds.
+
+## Core CI templates
+
+### pr-gate-build-job.yml
+
+This templates contains the jobs and most importantly the matrix of which packages and
+targets to run for Core CI.
+
+### pr-gate-steps.yml
+
+This template is the main Core CI template.  It controls all the steps run and is responsible for most functionality of the Core CI process.  This template sets
+the `pkg_count` variable using the `stuart_pr_eval` tool when the
+build type is "pull request"
+
+### spell-check-prereq-steps.yml
+
+This template installs the node based tools used by the spell checker plugin. The steps
+in this template are conditional and will only run if variable `pkg_count` is greater than 0.
+
+## Platform CI templates
+
+### platform-build-run-steps.yml
+
+This template makes heavy use of pytools to build and run a platform in the Edk2 repo
+
+Also uses basetools-build-steps.yml to compile basetools
+
+#### Special Notes
+
+* For a build type of pull request it will conditionally build if the patches change files that impact the platform.
+  * uses `stuart_pr_eval` to determine impact
+* For manual builds or CI builds it will always build the platform
+* It compiles basetools from source
+* Will use `stuart_build --FlashOnly` to attempt to run the built image if the `Run` parameter is set.
+* See the parameters block for expected configuration options
+* Parameter `extra_install_step` allows the caller to insert extra steps.  This is useful if additional dependencies, tools, or other things need to be installed.  Here is an example of installing qemu on Windows.
+
+    ``` yaml
+    steps:
+    - template: ../../.azurepipelines/templates/build-run-steps.yml
+      parameters:
+        extra_install_step:
+        - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
+          displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
+          condition: and(gt(variables.pkg_count, 0), succeeded())
+    ```

.azurepipelines/templates/platform-build-run-steps.yml

@@ -0,0 +1,134 @@
+
+## @file
+# File steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+- name: tool_chain_tag
+  type: string
+  default: ''
+- name: build_pkg
+  type: string
+  default: ''
+- name: build_target
+  type: string
+  default: ''
+- name: build_arch
+  type: string
+  default: ''
+- name: build_file
+  type: string
+  default: ''
+- name: build_flags
+  type: string
+  default: ''
+- name: run_flags
+  type: string
+  default: ''
+
+- name: extra_install_step
+  type: stepList
+  default: []
+
+steps:
+- checkout: self
+  clean: true
+  fetchDepth: 1
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: "3.8.x"
+    architecture: "x64"
+
+- script: pip install -r pip-requirements.txt --upgrade
+  displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+  displayName: Check if ${{ parameters.build_pkg }} need testing
+  inputs:
+    filename: stuart_pr_eval
+    arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+ # Setup repo
+- task: CmdLine@1
+  displayName: Setup
+  inputs:
+    filename: stuart_setup
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Stuart Update
+- task: CmdLine@1
+  displayName: Update
+  inputs:
+    filename: stuart_update
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# build basetools
+#   do this after setup and update so that code base dependencies
+#   are all resolved.
+- template: basetools-build-steps.yml
+  parameters:
+    tool_chain_tag: ${{ parameters.tool_chain_tag }}
+
+# Potential Extra steps
+- ${{ parameters.extra_install_step }}
+
+# Build
+- task: CmdLine@1
+  displayName: Build
+  inputs:
+    filename: stuart_build
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Run
+- task: CmdLine@1
+  displayName: Run to shell
+  inputs:
+    filename: stuart_build
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+  condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
+  timeoutInMinutes: 1
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+  displayName: "Copy build logs"
+  inputs:
+    targetFolder: "$(Build.ArtifactStagingDirectory)"
+    SourceFolder: "Build"
+    contents: |
+      BUILDLOG_*.txt
+      BUILDLOG_*.md
+      CI_*.txt
+      CI_*.md
+      CISETUP.txt
+      SETUPLOG.txt
+      UPDATE_LOG.txt
+      PREVALLOG.txt
+      TestSuites.xml
+      **/BUILD_TOOLS_REPORT.html
+      **/OVERRIDELOG.TXT
+      BASETOOLS_BUILD*.*
+    flattenFolders: true
+  condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+  continueOnError: true
+  displayName: "Publish build logs"
+  inputs:
+    pathtoPublish: "$(Build.ArtifactStagingDirectory)"
+    artifactName: "Build Logs $(System.JobName)"
+  condition: succeededOrFailed()

.azurepipelines/templates/pr-gate-build-job.yml

@@ -44,6 +44,11 @@ jobs:
       TARGET_SECURITY:
         Build.Pkgs: 'SecurityPkg'
         Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+      TARGET_PLATFORMS:
+        # For Platforms only check code. Leave it to Platform CI
+        # to build them.
+        Build.Pkgs: 'ArmVirtPkg,EmulatorPkg,OvmfPkg'
+        Build.Targets: 'NO-TARGET'
 
   workspace:
     clean: all

.azurepipelines/templates/pr-gate-steps.yml

@@ -20,7 +20,7 @@ steps:
 
 - task: UsePythonVersion@0
   inputs:
-    versionSpec: '3.7.x'
+    versionSpec: '3.8.x'
     architecture: 'x64'
 
 - script: pip install -r pip-requirements.txt --upgrade
@@ -39,11 +39,6 @@ steps:
     arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
   condition: eq(variables['Build.Reason'], 'PullRequest')
 
-# build basetools
-- template: basetools-build-steps.yml
-  parameters:
-    tool_chain_tag: ${{ parameters.tool_chain_tag }}
-
 # install spell check prereqs
 - template: spell-check-prereq-steps.yml
 
@@ -62,6 +57,13 @@ steps:
     arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
   condition: and(gt(variables.pkg_count, 0), succeeded())
 
+# build basetools
+#   do this after setup and update so that code base dependencies
+#   are all resolved.
+- template: basetools-build-steps.yml
+  parameters:
+    tool_chain_tag: ${{ parameters.tool_chain_tag }}
+
 - task: CmdLine@1
   displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
   inputs:

.gitmodules

@@ -7,3 +7,13 @@
 [submodule "UnitTestFrameworkPkg/Library/CmockaLib/cmocka"]
 	path = UnitTestFrameworkPkg/Library/CmockaLib/cmocka
 	url = https://git.cryptomilk.org/projects/cmocka.git
+[submodule "MdeModulePkg/Universal/RegularExpressionDxe/oniguruma"]
+	path = MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
+	url = https://github.com/kkos/oniguruma
+[submodule "MdeModulePkg/Library/BrotliCustomDecompressLib/brotli"]
+	path = MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
+	url = https://github.com/google/brotli
+[submodule "BaseTools/Source/C/BrotliCompress/brotli"]
+	path = BaseTools/Source/C/BrotliCompress/brotli
+	url = https://github.com/google/brotli
+	ignore = untracked

.mailmap

@@ -1,8 +1,7 @@
 #
-# This list is used by git-shortlog to fix a few botched name translations
-# in the git archive, either because the author's full name was messed up
-# and/or not always written the same way, making contributions from the
-# same person appearing not to be so or badly displayed.
+# This list is used by git-shortlog to update a few name translations
+# in the git archive to adjust for job changes or incorrect/inconsistent
+# name usage.
 #
 # Please keep this file sorted alphabetically, and email in lowercase.
 # The format used is:
@@ -23,6 +22,8 @@ Eric Dong <eric.dong@intel.com> Eric Dong <eirc.dong@intel.com>
 Eric Dong <eric.dong@intel.com> <ydong10@6f19259b-4bc3-4df7-8a09-765794883524>
 Eric Dong <eric.dong@intel.com> <ydong10@Edk2>
 Erik Bjorge <erik.c.bjorge@intel.com> <geekboy15a@6f19259b-4bc3-4df7-8a09-765794883524>
+Eugene Cohen <eugene@nuviainc.com>
+Eugene Cohen <eugene@nuviainc.com> <eugene@hp.com>
 Hao A Wu <hao.a.wu@intel.com>
 Hao A Wu <hao.a.wu@intel.com> <hwu1225@Edk2>
 Hot Tian <hot.tian@intel.com>

.pytool/CISettings.py

@@ -1,6 +1,7 @@
 # @file
 #
 # Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 ##
 import os
@@ -39,7 +40,9 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
         ''' return iterable of edk2 packages supported by this build.
         These should be edk2 workspace relative paths '''
 
-        return ("MdePkg",
+        return ("ArmVirtPkg",
+                "EmulatorPkg",
+                "MdePkg",
                 "MdeModulePkg",
                 "NetworkPkg",
                 "PcAtChipsetPkg",
@@ -49,15 +52,18 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
                 "ShellPkg",
                 "FatPkg",
                 "CryptoPkg",
-                "UnitTestFrameworkPkg"
+                "UnitTestFrameworkPkg",
+                "OvmfPkg"
                 )
 
     def GetArchitecturesSupported(self):
         ''' return iterable of edk2 architectures supported by this build '''
-        return ("IA32",
+        return (
+                "IA32",
                 "X64",
                 "ARM",
-                "AARCH64")
+                "AARCH64",
+                "RISCV64")
 
     def GetTargetsSupported(self):
         ''' return iterable of edk2 target tags supported by this build '''
@@ -122,14 +128,13 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
 
         self.ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
 
-        if GetHostInfo().os.upper() == "WINDOWS":
-            scopes += ('host-test-win',)
-
         if GetHostInfo().os.upper() == "LINUX" and self.ActualToolChainTag.upper().startswith("GCC"):
             if "AARCH64" in self.ActualArchitectures:
                 scopes += ("gcc_aarch64_linux",)
             if "ARM" in self.ActualArchitectures:
                 scopes += ("gcc_arm_linux",)
+            if "RISCV64" in self.ActualArchitectures:
+                scopes += ("gcc_riscv64_unknown",)
 
         return scopes
 
@@ -144,6 +149,12 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
             "CryptoPkg/Library/OpensslLib/openssl", False))
         rs.append(RequiredSubmodule(
             "UnitTestFrameworkPkg/Library/CmockaLib/cmocka", False))
+        rs.append(RequiredSubmodule(
+            "MdeModulePkg/Universal/RegularExpressionDxe/oniguruma", False))
+        rs.append(RequiredSubmodule(
+            "MdeModulePkg/Library/BrotliCustomDecompressLib/brotli", False))
+        rs.append(RequiredSubmodule(
+            "BaseTools/Source/C/BrotliCompress/brotli", False))
         return rs
 
     def GetName(self):

.pytool/Plugin/SpellCheck/cspell.base.yaml

@@ -22,6 +22,8 @@
     ],
     "minWordLength": 5,
     "allowCompoundWords": false,
+    "maxNumberOfProblems": 200,
+    "maxDuplicateProblems": 200,
     "ignoreWords": [
         "muchange"
     ],
@@ -161,5 +163,20 @@
         "bootability",
         "Sdhci",
         "inmodule",
+        "RISCV",
+        "edksetup",
+        "iscsi",
+        "nvdata",
+        "pytools",
+        "NTDDI",
+        "Wnonportable",
+        "CLANGPDB",
+        "nologo",
+        "lldmap",
+        "ASMLINK",
+        "NODEFAULTLIB",
+        "vcruntimed",
+        "ucrtd",
+        "msvcrtd",
     ]
 }

.pytool/Readme.md

@@ -2,31 +2,32 @@
 
 ## Basic Status
 
-| Package             | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |
-| :----               | :-----                   | :----                             | :---         |
-| ArmPkg              |
-| ArmPlatformPkg      |
-| ArmVirtPkg          |
-| CryptoPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
-| DynamicTablesPkg    |
-| EmbeddedPkg         |
-| EmulatorPkg         |
-| FatPkg              | :heavy_check_mark: | :heavy_check_mark: |
-| FmpDevicePkg        | :heavy_check_mark: | :heavy_check_mark: |
-| IntelFsp2Pkg        |
-| IntelFsp2WrapperPkg |
-| MdeModulePkg        | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode
-| MdePkg              | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
-| NetworkPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
-| OvmfPkg             |
-| PcAtChipsetPkg      | :heavy_check_mark: | :heavy_check_mark: |
-| SecurityPkg         | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
-| ShellPkg            | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC
-| SignedCapsulePkg    |
-| SourceLevelDebugPkg |
-| StandaloneMmPkg     |
-| UefiCpuPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC
-| UefiPayloadPkg      |
+| Package              | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |
+| :----                | :-----                   | :----                             | :---         |
+| ArmPkg               |
+| ArmPlatformPkg       |
+| ArmVirtPkg           | SEE PACKAGE README | SEE PACKAGE README |
+| CryptoPkg            | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
+| DynamicTablesPkg     |
+| EmbeddedPkg          |
+| EmulatorPkg          | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode
+| FatPkg               | :heavy_check_mark: | :heavy_check_mark: |
+| FmpDevicePkg         | :heavy_check_mark: | :heavy_check_mark: |
+| IntelFsp2Pkg         |
+| IntelFsp2WrapperPkg  |
+| MdeModulePkg         | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode
+| MdePkg               | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
+| NetworkPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
+| OvmfPkg              | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode
+| PcAtChipsetPkg       | :heavy_check_mark: | :heavy_check_mark: |
+| SecurityPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
+| ShellPkg             | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC
+| SignedCapsulePkg     |
+| SourceLevelDebugPkg  |
+| StandaloneMmPkg      |
+| UefiCpuPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC
+| UefiPayloadPkg       |
+| UnitTestFrameworkPkg | :heavy_check_mark: | :heavy_check_mark: |
 
 For more detailed status look at the test results of the latest CI run on the
 repo readme.
@@ -77,7 +78,7 @@ per package configuration which comes from this file.
 ## Running CI locally
 
 The EDKII Tools environment (and by extension the ci) is designed to support
-easily and consistantly running locally and in a cloud ci environment.  To do
+easily and consistently running locally and in a cloud ci environment.  To do
 that a few steps should be followed.  Details of EDKII Tools can be found in the
 [docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)
 
@@ -88,7 +89,7 @@ that a few steps should be followed.  Details of EDKII Tools can be found in the
      * VS 2017 or VS 2019
      * Windows SDK (for rc)
      * Windows WDK (for capsules)
-   * Ubuntu 16.04
+   * Ubuntu 18.04 or Fedora
      * GCC5
    * Easy to add more but this is the current state
 2. Python 3.7.x or newer on path
@@ -137,11 +138,31 @@ location makes more sense for the community.
 
 ### Module Inclusion Test - DscCompleteCheck
 
-This test scans all available modules (via INF files) and compares them to the
-package-level DSC file for the package each module is contained within. The test
-considers it an error if any module does not appear in the `Components` section
-of at least one package-level DSC (indicating that it would not be built if the
-package were built).
+This scans all INF files from a package and confirms they are
+listed in the package level DSC file. The test considers it an error if any INF
+does not appear in the `Components` section of the package-level DSC (indicating
+that it would not be built if the package were built). This is critical because
+much of the CI infrastructure assumes that all modules will be listed in the DSC
+and compiled.
+
+This test will ignore INFs in the following cases:
+
+1. When `MODULE_TYPE` = `HOST_APPLICATION`
+2. When a Library instance **only** supports the `HOST_APPLICATION` environment
+
+### Host Module Inclusion Test - HostUnitTestDscCompleteCheck
+
+This test scans all INF files from a package for those related to host
+based unit tests and confirms they are listed in the unit test DSC file for the package.
+The test considers it an error if any INF meeting the requirements does not appear
+in the `Components` section of the unit test DSC. This is critical because
+much of the CI infrastructure assumes that  modules will be listed in the DSC
+and compiled.
+
+This test will only require INFs in the following cases:
+
+1. When `MODULE_TYPE` = `HOST_APPLICATION`
+2. When a Library instance explicitly supports the `HOST_APPLICATION` environment
 
 ### Code Compilation Test - CompilerPlugin
 
@@ -150,6 +171,46 @@ all package-level DSCs were built, the Code Compilation Test simply runs through
 and builds every package-level DSC on every toolchain and for every architecture
 that is supported. Any module that fails to build is considered an error.
 
+### Host Unit Test Compilation and Run Test - HostUnitTestCompilerPlugin
+
+A test that compiles the dsc for host based unit test apps.
+On Windows this will also enable a build plugin to execute that will run the unit tests and verify the results.
+
+These tools will be invoked on any CI
+pass that includes the NOOPT target. In order for these tools to do their job,
+the package and tests must be configured in a particular way...
+
+#### Including Host-Based Tests in the Package YAML
+
+For example, looking at the `MdeModulePkg.ci.yaml` config file, there are two
+config options that control HostBased test behavior:
+
+```json
+    ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+    "HostUnitTestCompilerPlugin": {
+        "DscPath": "Test/MdeModulePkgHostTest.dsc"
+    },
+```
+
+This option tell the test builder to run. The test builder needs to know which
+modules in this package are host-based tests, so that DSC path is provided.
+
+#### Configuring the HostBased DSC
+
+The HostBased DSC for `MdeModulePkg` is located at
+`MdeModulePkg/Test/MdeModulePkgHostTest.dsc`.
+
+To add automated host-based unit test building to a new package, create a
+similar DSC. The new DSC should make sure to have the `NOOPT` BUILD_TARGET
+and should include the line:
+
+```
+!include UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
+```
+
+All of the modules that are included in the `Components` section of this
+DSC should be of type HOST_APPLICATION.
+
 ### GUID Uniqueness Test - GuidCheck
 
 This test works on the collection of all packages rather than an individual
@@ -207,6 +268,8 @@ few standard scopes.
 | global-nix | edk2_invocable++                                     | Running on Linux based OS    |
 | edk2-build |                                                      | This indicates that an invocable is building EDK2 based UEFI code |
 | cibuild    | set in .pytool/CISettings.py                         | Suggested target for edk2 continuous integration builds.  Tools used for CiBuilds can use this scope.  Example: asl compiler |
+| host-based-test | set in .pytool/CISettings.py                    | Turns on the host based tests and plugin |
+| host-test-win | set in .pytool/CISettings.py                      | Enables the host based test runner for Windows |
 
 ## Future investments
 
@@ -216,8 +279,6 @@ few standard scopes.
 * Visual Studio AARCH64 and ARM support
 * BaseTools C tools CI/PR and binary release process
 * BaseTools Python tools CI/PR process
-* Host based unit testing
 * Extensible private/closed source platform reporting
-* Platform builds, validation
 * UEFI SCTs
 * Other automation

ArmPkg/ArmPkg.dsc

@@ -4,6 +4,7 @@
 # Copyright (c) 2009 - 2010, Apple Inc. All rights reserved.<BR>
 # Copyright (c) 2011 - 2018, ARM Ltd. All rights reserved.<BR>
 # Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
+# Copyright (c) Microsoft Corporation.<BR>
 #
 #    SPDX-License-Identifier: BSD-2-Clause-Patent
 #
@@ -133,11 +134,15 @@
   ArmPkg/Library/ArmGicArchLib/ArmGicArchLib.inf
   ArmPkg/Library/ArmGicArchSecLib/ArmGicArchSecLib.inf
   ArmPkg/Library/ArmLib/ArmBaseLib.inf
+  ArmPkg/Library/ArmMtlNullLib/ArmMtlNullLib.inf
   ArmPkg/Library/ArmSoftFloatLib/ArmSoftFloatLib.inf
+  ArmPkg/Library/ArmSmcPsciResetSystemLib/ArmSmcPsciResetSystemLib.inf
   ArmPkg/Library/PeiServicesTablePointerLib/PeiServicesTablePointerLib.inf
   ArmPkg/Library/PlatformBootManagerLib/PlatformBootManagerLib.inf
 
+  ArmPkg/Drivers/ArmCrashDumpDxe/ArmCrashDumpDxe.inf
   ArmPkg/Drivers/ArmScmiDxe/ArmScmiDxe.inf
+  ArmPkg/Drivers/MmCommunicationDxe/MmCommunication.inf
 
 [Components.AARCH64]
   ArmPkg/Library/ArmMmuLib/ArmMmuPeiLib.inf

ArmPkg/Drivers/CpuDxe/AArch64/Mmu.c

@@ -13,7 +13,68 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
 #include <Library/MemoryAllocationLib.h>
 #include "CpuDxe.h"
 
-#define TT_ATTR_INDX_INVALID    ((UINT32)~0)
+#define INVALID_ENTRY   ((UINT32)~0)
+
+#define MIN_T0SZ        16
+#define BITS_PER_LEVEL  9
+
+STATIC
+VOID
+GetRootTranslationTableInfo (
+  IN  UINTN     T0SZ,
+  OUT UINTN     *RootTableLevel,
+  OUT UINTN     *RootTableEntryCount
+  )
+{
+  *RootTableLevel       = (T0SZ - MIN_T0SZ) / BITS_PER_LEVEL;
+  *RootTableEntryCount  = TT_ENTRY_COUNT >> (T0SZ - MIN_T0SZ) % BITS_PER_LEVEL;
+}
+
+STATIC
+UINT64
+PageAttributeToGcdAttribute (
+  IN UINT64 PageAttributes
+  )
+{
+  UINT64  GcdAttributes;
+
+  switch (PageAttributes & TT_ATTR_INDX_MASK) {
+  case TT_ATTR_INDX_DEVICE_MEMORY:
+    GcdAttributes = EFI_MEMORY_UC;
+    break;
+  case TT_ATTR_INDX_MEMORY_NON_CACHEABLE:
+    GcdAttributes = EFI_MEMORY_WC;
+    break;
+  case TT_ATTR_INDX_MEMORY_WRITE_THROUGH:
+    GcdAttributes = EFI_MEMORY_WT;
+    break;
+  case TT_ATTR_INDX_MEMORY_WRITE_BACK:
+    GcdAttributes = EFI_MEMORY_WB;
+    break;
+  default:
+    DEBUG ((DEBUG_ERROR,
+      "PageAttributeToGcdAttribute: PageAttributes:0x%lX not supported.\n",
+      PageAttributes));
+    ASSERT (0);
+    // The Global Coherency Domain (GCD) value is defined as a bit set.
+    // Returning 0 means no attribute has been set.
+    GcdAttributes = 0;
+  }
+
+  // Determine protection attributes
+  if (((PageAttributes & TT_AP_MASK) == TT_AP_NO_RO) ||
+      ((PageAttributes & TT_AP_MASK) == TT_AP_RO_RO)) {
+    // Read only cases map to write-protect
+    GcdAttributes |= EFI_MEMORY_RO;
+  }
+
+  // Process eXecute Never attribute
+  if ((PageAttributes & (TT_PXN_MASK | TT_UXN_MASK)) != 0) {
+    GcdAttributes |= EFI_MEMORY_XP;
+  }
+
+  return GcdAttributes;
+}
 
 STATIC
 UINT64
@@ -37,7 +98,7 @@ GetFirstPageAttribute (
   {
     return FirstEntry & TT_ATTR_INDX_MASK;
   } else {
-    return TT_ATTR_INDX_INVALID;
+    return INVALID_ENTRY;
   }
 }
 
@@ -78,8 +139,8 @@ GetNextEntryAttribute (
     // If Entry is a Table Descriptor type entry then go through the sub-level table
     if ((EntryType == TT_TYPE_BLOCK_ENTRY) ||
         ((TableLevel == 3) && (EntryType == TT_TYPE_BLOCK_ENTRY_LEVEL3))) {
-      if ((*PrevEntryAttribute == TT_ATTR_INDX_INVALID) || (EntryAttribute != *PrevEntryAttribute)) {
-        if (*PrevEntryAttribute != TT_ATTR_INDX_INVALID) {
+      if ((*PrevEntryAttribute == INVALID_ENTRY) || (EntryAttribute != *PrevEntryAttribute)) {
+        if (*PrevEntryAttribute != INVALID_ENTRY) {
           // Update GCD with the last region
           SetGcdMemorySpaceAttributes (MemorySpaceMap, NumberOfDescriptors,
               *StartGcdRegion,
@@ -103,7 +164,7 @@ GetNextEntryAttribute (
                              (BaseAddress + (Index * TT_ADDRESS_AT_LEVEL(TableLevel))),
                              PrevEntryAttribute, StartGcdRegion);
     } else {
-      if (*PrevEntryAttribute != TT_ATTR_INDX_INVALID) {
+      if (*PrevEntryAttribute != INVALID_ENTRY) {
         // Update GCD with the last region
         SetGcdMemorySpaceAttributes (MemorySpaceMap, NumberOfDescriptors,
             *StartGcdRegion,
@@ -112,7 +173,7 @@ GetNextEntryAttribute (
 
         // Start of the new region
         *StartGcdRegion = BaseAddress + (Index * TT_ADDRESS_AT_LEVEL(TableLevel));
-        *PrevEntryAttribute = TT_ATTR_INDX_INVALID;
+        *PrevEntryAttribute = INVALID_ENTRY;
       }
     }
   }
@@ -177,7 +238,7 @@ SyncCacheConfig (
                                                &PageAttribute, &BaseAddressGcdRegion);
 
   // Update GCD with the last region if valid
-  if (PageAttribute != TT_ATTR_INDX_INVALID) {
+  if (PageAttribute != INVALID_ENTRY) {
     SetGcdMemorySpaceAttributes (MemorySpaceMap, NumberOfDescriptors,
         BaseAddressGcdRegion,
         EndAddressGcdRegion - BaseAddressGcdRegion,

ArmPkg/Drivers/CpuDxe/CpuDxe.h

@@ -134,13 +134,6 @@ GetMemoryRegion (
   OUT    UINTN                   *RegionAttributes
   );
 
-VOID
-GetRootTranslationTableInfo (
-  IN  UINTN    T0SZ,
-  OUT UINTN   *TableLevel,
-  OUT UINTN   *TableEntryCount
-  );
-
 EFI_STATUS
 SetGcdMemorySpaceAttributes (
   IN EFI_GCD_MEMORY_SPACE_DESCRIPTOR    *MemorySpaceMap,

ArmPkg/Drivers/MmCommunicationDxe/MmCommunication.c

@@ -1,6 +1,6 @@
 /** @file
 
-  Copyright (c) 2016-2018, ARM Limited. All rights reserved.
+  Copyright (c) 2016-2019, ARM Limited. All rights reserved.
 
   SPDX-License-Identifier: BSD-2-Clause-Patent
 
@@ -16,7 +16,7 @@
 #include <Library/UefiBootServicesTableLib.h>
 #include <Library/UefiRuntimeServicesTableLib.h>
 
-#include <Protocol/MmCommunication.h>
+#include <Protocol/MmCommunication2.h>
 
 #include <IndustryStandard/ArmStdSmc.h>
 
@@ -39,39 +39,34 @@ STATIC EFI_HANDLE  mMmCommunicateHandle;
 /**
   Communicates with a registered handler.
 
-  This function provides an interface to send and receive messages to the
-  Standalone MM environment on behalf of UEFI services.  This function is part
-  of the MM Communication Protocol that may be called in physical mode prior to
-  SetVirtualAddressMap() and in virtual mode after SetVirtualAddressMap().
+  This function provides a service to send and receive messages from a registered UEFI service.
 
-  @param[in]      This                The EFI_MM_COMMUNICATION_PROTOCOL
-                                      instance.
-  @param[in, out] CommBuffer          A pointer to the buffer to convey
-                                      into MMRAM.
-  @param[in, out] CommSize            The size of the data buffer being
-                                      passed in. This is optional.
+  @param[in] This                The EFI_MM_COMMUNICATION_PROTOCOL instance.
+  @param[in] CommBufferPhysical  Physical address of the MM communication buffer
+  @param[in] CommBufferVirtual   Virtual address of the MM communication buffer
+  @param[in] CommSize            The size of the data buffer being passed in. On exit, the size of data
+                                 being returned. Zero if the handler does not wish to reply with any data.
+                                 This parameter is optional and may be NULL.
+
+  @retval EFI_SUCCESS            The message was successfully posted.
+  @retval EFI_INVALID_PARAMETER  CommBufferPhysical was NULL or CommBufferVirtual was NULL.
+  @retval EFI_BAD_BUFFER_SIZE    The buffer is too large for the MM implementation.
+                                 If this error is returned, the MessageLength field
+                                 in the CommBuffer header or the integer pointed by
+                                 CommSize, are updated to reflect the maximum payload
+                                 size the implementation can accommodate.
+  @retval EFI_ACCESS_DENIED      The CommunicateBuffer parameter or CommSize parameter,
+                                 if not omitted, are in address range that cannot be
+                                 accessed by the MM environment.
 
-  @retval EFI_SUCCESS                 The message was successfully posted.
-  @retval EFI_INVALID_PARAMETER       The CommBuffer was NULL.
-  @retval EFI_BAD_BUFFER_SIZE         The buffer size is incorrect for the MM
-                                      implementation. If this error is
-                                      returned, the MessageLength field in
-                                      the CommBuffer header or the integer
-                                      pointed by CommSize are updated to reflect
-                                      the maximum payload size the
-                                      implementation can accommodate.
-  @retval EFI_ACCESS_DENIED           The CommunicateBuffer parameter
-                                      or CommSize parameter, if not omitted,
-                                      are in address range that cannot be
-                                      accessed by the MM environment
 **/
-STATIC
 EFI_STATUS
 EFIAPI
-MmCommunicationCommunicate (
-  IN CONST EFI_MM_COMMUNICATION_PROTOCOL  *This,
-  IN OUT VOID                             *CommBuffer,
-  IN OUT UINTN                            *CommSize OPTIONAL
+MmCommunication2Communicate (
+  IN CONST EFI_MM_COMMUNICATION2_PROTOCOL   *This,
+  IN OUT VOID                               *CommBufferPhysical,
+  IN OUT VOID                               *CommBufferVirtual,
+  IN OUT UINTN                              *CommSize OPTIONAL
   )
 {
   EFI_MM_COMMUNICATE_HEADER   *CommunicateHeader;
@@ -87,11 +82,11 @@ MmCommunicationCommunicate (
   //
   // Check parameters
   //
-  if (CommBuffer == NULL) {
+  if (CommBufferVirtual == NULL) {
     return EFI_INVALID_PARAMETER;
   }
 
-  CommunicateHeader = CommBuffer;
+  CommunicateHeader = CommBufferVirtual;
   // CommBuffer is a mandatory parameter. Hence, Rely on
   // MessageLength + Header to ascertain the
   // total size of the communication payload rather than
@@ -136,7 +131,7 @@ MmCommunicationCommunicate (
   CommunicateSmcArgs.Arg1 = 0;
 
   // Copy Communication Payload
-  CopyMem ((VOID *)mNsCommBuffMemRegion.VirtualBase, CommBuffer, BufferSize);
+  CopyMem ((VOID *)mNsCommBuffMemRegion.VirtualBase, CommBufferVirtual, BufferSize);
 
   // comm_buffer_address (64-bit physical address)
   CommunicateSmcArgs.Arg2 = (UINTN)mNsCommBuffMemRegion.PhysicalBase;
@@ -149,7 +144,7 @@ MmCommunicationCommunicate (
 
   switch (CommunicateSmcArgs.Arg0) {
   case ARM_SMC_MM_RET_SUCCESS:
-    ZeroMem (CommBuffer, BufferSize);
+    ZeroMem (CommBufferVirtual, BufferSize);
     // On successful return, the size of data being returned is inferred from
     // MessageLength + Header.
     CommunicateHeader = (EFI_MM_COMMUNICATE_HEADER *)mNsCommBuffMemRegion.VirtualBase;
@@ -158,7 +153,7 @@ MmCommunicationCommunicate (
                  sizeof (CommunicateHeader->MessageLength);
 
     CopyMem (
-      CommBuffer,
+      CommBufferVirtual,
       (VOID *)mNsCommBuffMemRegion.VirtualBase,
       BufferSize
       );
@@ -191,8 +186,8 @@ MmCommunicationCommunicate (
 //
 // MM Communication Protocol instance
 //
-EFI_MM_COMMUNICATION_PROTOCOL  mMmCommunication = {
-  MmCommunicationCommunicate
+STATIC EFI_MM_COMMUNICATION2_PROTOCOL  mMmCommunication2 = {
+  MmCommunication2Communicate
 };
 
 /**
@@ -293,7 +288,7 @@ MmGuidedEventNotify (
   Header.Data[0] = 0;
 
   Size = sizeof (Header);
-  MmCommunicationCommunicate (&mMmCommunication, &Header, &Size);
+  MmCommunication2Communicate (&mMmCommunication2, &Header, &Header, &Size);
 }
 
 /**
@@ -312,7 +307,7 @@ MmGuidedEventNotify (
 **/
 EFI_STATUS
 EFIAPI
-MmCommunicationInitialize (
+MmCommunication2Initialize (
   IN EFI_HANDLE         ImageHandle,
   IN EFI_SYSTEM_TABLE  *SystemTable
   )
@@ -363,9 +358,9 @@ MmCommunicationInitialize (
   // Install the communication protocol
   Status = gBS->InstallProtocolInterface (
                   &mMmCommunicateHandle,
-                  &gEfiMmCommunicationProtocolGuid,
+                  &gEfiMmCommunication2ProtocolGuid,
                   EFI_NATIVE_INTERFACE,
-                  &mMmCommunication
+                  &mMmCommunication2
                   );
   if (EFI_ERROR(Status)) {
     DEBUG ((DEBUG_ERROR, "MmCommunicationInitialize: "
@@ -390,12 +385,20 @@ MmCommunicationInitialize (
                     MmGuidedEventNotify, mGuidedEventGuid[Index],
                     mGuidedEventGuid[Index], &mGuidedEvent[Index]);
     ASSERT_EFI_ERROR (Status);
+    if (EFI_ERROR (Status)) {
+      while (Index-- > 0) {
+        gBS->CloseEvent (mGuidedEvent[Index]);
+      }
+      goto UninstallProtocol;
+    }
   }
+  return EFI_SUCCESS;
 
+UninstallProtocol:
   gBS->UninstallProtocolInterface (
          mMmCommunicateHandle,
-         &gEfiMmCommunicationProtocolGuid,
-         &mMmCommunication
+         &gEfiMmCommunication2ProtocolGuid,
+         &mMmCommunication2
          );
 
 CleanAddedMemorySpace:

ArmPkg/Drivers/MmCommunicationDxe/MmCommunication.inf

@@ -2,7 +2,7 @@
 #
 #  DXE MM Communicate driver
 #
-#  Copyright (c) 2016 - 2018, ARM Limited. All rights reserved.
+#  Copyright (c) 2016 - 2019, ARM Limited. All rights reserved.
 #
 #  SPDX-License-Identifier: BSD-2-Clause-Patent
 #
@@ -14,7 +14,7 @@
   FILE_GUID                      = 09EE81D3-F15E-43F4-85B4-CB9873DA5D6B
   MODULE_TYPE                    = DXE_RUNTIME_DRIVER
   VERSION_STRING                 = 1.0
-  ENTRY_POINT                    = MmCommunicationInitialize
+  ENTRY_POINT                    = MmCommunication2Initialize
 
 #
 # The following is for reference only and not required by
@@ -40,7 +40,7 @@
   UefiDriverEntryPoint
 
 [Protocols]
-  gEfiMmCommunicationProtocolGuid              ## PRODUCES
+  gEfiMmCommunication2ProtocolGuid              ## PRODUCES
 
 [Guids]
   gEfiEndOfDxeEventGroupGuid

ArmPkg/Include/Chipset/AArch64.h

@@ -219,11 +219,6 @@ ArmReadCurrentEL (
   VOID
   );
 
-UINT64
-PageAttributeToGcdAttribute (
-  IN UINT64 PageAttributes
-  );
-
 UINTN
 ArmWriteCptr (
   IN  UINT64 Cptr

ArmPkg/Include/Library/ArmLib.h

@@ -211,24 +211,6 @@ ArmCleanInvalidateDataCacheEntryByMVA (
   IN  UINTN   Address
   );
 
-VOID
-EFIAPI
-ArmInvalidateDataCacheEntryBySetWay (
-  IN  UINTN  SetWayFormat
-  );
-
-VOID
-EFIAPI
-ArmCleanDataCacheEntryBySetWay (
-  IN  UINTN  SetWayFormat
-  );
-
-VOID
-EFIAPI
-ArmCleanInvalidateDataCacheEntryBySetWay (
-  IN  UINTN   SetWayFormat
-  );
-
 VOID
 EFIAPI
 ArmEnableDataCache (

ArmPkg/Library/ArmLib/AArch64/AArch64Lib.c

@@ -7,11 +7,13 @@
 
 **/
 
-#include <Uefi.h>
-#include <Chipset/AArch64.h>
+#include <Base.h>
+
 #include <Library/ArmLib.h>
-#include <Library/BaseLib.h>
-#include <Library/IoLib.h>
+#include <Library/DebugLib.h>
+
+#include <Chipset/AArch64.h>
+
 #include "AArch64Lib.h"
 #include "ArmLibPrivate.h"
 
@@ -40,6 +42,8 @@ ArmInvalidateDataCache (
   VOID
   )
 {
+  ASSERT (!ArmMmuEnabled ());
+
   ArmDataSynchronizationBarrier ();
   AArch64DataCacheOperation (ArmInvalidateDataCacheEntryBySetWay);
 }
@@ -50,6 +54,8 @@ ArmCleanInvalidateDataCache (
   VOID
   )
 {
+  ASSERT (!ArmMmuEnabled ());
+
   ArmDataSynchronizationBarrier ();
   AArch64DataCacheOperation (ArmCleanInvalidateDataCacheEntryBySetWay);
 }
@@ -60,6 +66,8 @@ ArmCleanDataCache (
   VOID
   )
 {
+  ASSERT (!ArmMmuEnabled ());
+
   ArmDataSynchronizationBarrier ();
   AArch64DataCacheOperation (ArmCleanDataCacheEntryBySetWay);
 }

ArmPkg/Library/ArmLib/AArch64/AArch64Lib.h

@@ -17,5 +17,23 @@ AArch64AllDataCachesOperation (
   IN  AARCH64_CACHE_OPERATION  DataCacheOperation
   );
 
+VOID
+EFIAPI
+ArmInvalidateDataCacheEntryBySetWay (
+  IN  UINTN  SetWayFormat
+  );
+
+VOID
+EFIAPI
+ArmCleanDataCacheEntryBySetWay (
+  IN  UINTN  SetWayFormat
+  );
+
+VOID
+EFIAPI
+ArmCleanInvalidateDataCacheEntryBySetWay (
+  IN  UINTN   SetWayFormat
+  );
+
 #endif // __AARCH64_LIB_H__
 

ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S

@@ -13,6 +13,8 @@
 .set DAIF_RD_FIQ_BIT,   (1 << 6)
 .set DAIF_RD_IRQ_BIT,   (1 << 7)
 
+.set SCTLR_ELx_M_BIT_POS, (0)
+
 ASM_FUNC(ArmReadMidr)
   mrs     x0, midr_el1        // Read from Main ID Register (MIDR)
   ret
@@ -120,13 +122,18 @@ ASM_FUNC(ArmSetMAIR)
 ASM_FUNC(ArmUpdateTranslationTableEntry)
    dsb     nshst
    lsr     x1, x1, #12
-   EL1_OR_EL2_OR_EL3(x0)
+   EL1_OR_EL2_OR_EL3(x2)
 1: tlbi    vaae1, x1             // TLB Invalidate VA , EL1
+   mrs     x2, sctlr_el1
    b       4f
 2: tlbi    vae2, x1              // TLB Invalidate VA , EL2
+   mrs     x2, sctlr_el2
    b       4f
 3: tlbi    vae3, x1              // TLB Invalidate VA , EL3
-4: dsb     nsh
+   mrs     x2, sctlr_el3
+4: tbnz    x2, SCTLR_ELx_M_BIT_POS, 5f
+   dc      ivac, x0              // invalidate in Dcache if MMU is still off
+5: dsb     nsh
    isb
    ret
 

ArmPkg/Library/ArmLib/Arm/ArmV7Lib.c

@@ -6,11 +6,14 @@
   SPDX-License-Identifier: BSD-2-Clause-Patent
 
 **/
-#include <Uefi.h>
-#include <Chipset/ArmV7.h>
+
+#include <Base.h>
+
 #include <Library/ArmLib.h>
-#include <Library/BaseLib.h>
-#include <Library/IoLib.h>
+#include <Library/DebugLib.h>
+
+#include <Chipset/ArmV7.h>
+
 #include "ArmV7Lib.h"
 #include "ArmLibPrivate.h"
 
@@ -39,6 +42,8 @@ ArmInvalidateDataCache (
   VOID
   )
 {
+  ASSERT (!ArmMmuEnabled ());
+
   ArmDataSynchronizationBarrier ();
   ArmV7DataCacheOperation (ArmInvalidateDataCacheEntryBySetWay);
 }
@@ -49,6 +54,8 @@ ArmCleanInvalidateDataCache (
   VOID
   )
 {
+  ASSERT (!ArmMmuEnabled ());
+
   ArmDataSynchronizationBarrier ();
   ArmV7DataCacheOperation (ArmCleanInvalidateDataCacheEntryBySetWay);
 }
@@ -59,6 +66,8 @@ ArmCleanDataCache (
   VOID
   )
 {
+  ASSERT (!ArmMmuEnabled ());
+
   ArmDataSynchronizationBarrier ();
   ArmV7DataCacheOperation (ArmCleanDataCacheEntryBySetWay);
 }

ArmPkg/Library/ArmLib/Arm/ArmV7Lib.h

@@ -30,5 +30,23 @@ ArmV7AllDataCachesOperation (
   IN  ARM_V7_CACHE_OPERATION  DataCacheOperation
   );
 
+VOID
+EFIAPI
+ArmInvalidateDataCacheEntryBySetWay (
+  IN  UINTN  SetWayFormat
+  );
+
+VOID
+EFIAPI
+ArmCleanDataCacheEntryBySetWay (
+  IN  UINTN  SetWayFormat
+  );
+
+VOID
+EFIAPI
+ArmCleanInvalidateDataCacheEntryBySetWay (
+  IN  UINTN   SetWayFormat
+  );
+
 #endif // __ARM_V7_LIB_H__
 

ArmPkg/Library/ArmLib/ArmBaseLib.inf

@@ -44,12 +44,12 @@
   AArch64/AArch64Support.S
   AArch64/AArch64ArchTimerSupport.S
 
+[LibraryClasses]
+  DebugLib
+
 [Packages]
   ArmPkg/ArmPkg.dec
   MdePkg/MdePkg.dec
 
-[Protocols]
-  gEfiCpuArchProtocolGuid
-
 [FeaturePcd.ARM]
   gArmTokenSpaceGuid.PcdNormalMemoryNonshareableOverride

ArmPkg/Library/ArmLib/ArmLib.c

@@ -10,8 +10,6 @@
 #include <Base.h>
 
 #include <Library/ArmLib.h>
-#include <Library/DebugLib.h>
-#include <Library/PcdLib.h>
 
 #include "ArmLibPrivate.h"
 

ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c

@@ -1,7 +1,7 @@
 /** @file
 *  File managing the MMU for ARMv8 architecture
 *
-*  Copyright (c) 2011-2014, ARM Limited. All rights reserved.
+*  Copyright (c) 2011-2020, ARM Limited. All rights reserved.
 *  Copyright (c) 2016, Linaro Limited. All rights reserved.
 *  Copyright (c) 2017, Intel Corporation. All rights reserved.<BR>
 *
@@ -19,9 +19,6 @@
 #include <Library/BaseLib.h>
 #include <Library/DebugLib.h>
 
-// We use this index definition to define an invalid block entry
-#define TT_ATTR_INDX_INVALID    ((UINT32)~0)
-
 STATIC
 UINT64
 ArmMemoryAttributeToPageAttribute (
@@ -47,7 +44,7 @@ ArmMemoryAttributeToPageAttribute (
     return TT_ATTR_INDX_MEMORY_NON_CACHEABLE;
 
   default:
-    ASSERT(0);
+    ASSERT (0);
   case ARM_MEMORY_REGION_ATTRIBUTE_DEVICE:
   case ARM_MEMORY_REGION_ATTRIBUTE_NONSECURE_DEVICE:
     if (ArmReadCurrentEL () == AARCH64_EL2)
@@ -57,78 +54,40 @@ ArmMemoryAttributeToPageAttribute (
   }
 }
 
-UINT64
-PageAttributeToGcdAttribute (
-  IN UINT64 PageAttributes
-  )
-{
-  UINT64  GcdAttributes;
-
-  switch (PageAttributes & TT_ATTR_INDX_MASK) {
-  case TT_ATTR_INDX_DEVICE_MEMORY:
-    GcdAttributes = EFI_MEMORY_UC;
-    break;
-  case TT_ATTR_INDX_MEMORY_NON_CACHEABLE:
-    GcdAttributes = EFI_MEMORY_WC;
-    break;
-  case TT_ATTR_INDX_MEMORY_WRITE_THROUGH:
-    GcdAttributes = EFI_MEMORY_WT;
-    break;
-  case TT_ATTR_INDX_MEMORY_WRITE_BACK:
-    GcdAttributes = EFI_MEMORY_WB;
-    break;
-  default:
-    DEBUG ((EFI_D_ERROR, "PageAttributeToGcdAttribute: PageAttributes:0x%lX not supported.\n", PageAttributes));
-    ASSERT (0);
-    // The Global Coherency Domain (GCD) value is defined as a bit set.
-    // Returning 0 means no attribute has been set.
-    GcdAttributes = 0;
-  }
-
-  // Determine protection attributes
-  if (((PageAttributes & TT_AP_MASK) == TT_AP_NO_RO) || ((PageAttributes & TT_AP_MASK) == TT_AP_RO_RO)) {
-    // Read only cases map to write-protect
-    GcdAttributes |= EFI_MEMORY_RO;
-  }
-
-  // Process eXecute Never attribute
-  if ((PageAttributes & (TT_PXN_MASK | TT_UXN_MASK)) != 0 ) {
-    GcdAttributes |= EFI_MEMORY_XP;
-  }
-
-  return GcdAttributes;
-}
-
 #define MIN_T0SZ        16
 #define BITS_PER_LEVEL  9
+#define MAX_VA_BITS     48
 
-VOID
-GetRootTranslationTableInfo (
-  IN UINTN     T0SZ,
-  OUT UINTN   *TableLevel,
-  OUT UINTN   *TableEntryCount
+STATIC
+UINTN
+GetRootTableEntryCount (
+  IN  UINTN T0SZ
   )
 {
-  // Get the level of the root table
-  if (TableLevel) {
-    *TableLevel = (T0SZ - MIN_T0SZ) / BITS_PER_LEVEL;
-  }
+  return TT_ENTRY_COUNT >> (T0SZ - MIN_T0SZ) % BITS_PER_LEVEL;
+}
 
-  if (TableEntryCount) {
-    *TableEntryCount = 1UL << (BITS_PER_LEVEL - (T0SZ - MIN_T0SZ) % BITS_PER_LEVEL);
-  }
+STATIC
+UINTN
+GetRootTableLevel (
+  IN  UINTN T0SZ
+  )
+{
+  return (T0SZ - MIN_T0SZ) / BITS_PER_LEVEL;
 }
 
 STATIC
 VOID
-ReplaceLiveEntry (
+ReplaceTableEntry (
   IN  UINT64  *Entry,
   IN  UINT64  Value,
-  IN  UINT64  RegionStart
+  IN  UINT64  RegionStart,
+  IN  BOOLEAN IsLiveBlockMapping
   )
 {
-  if (!ArmMmuEnabled ()) {
+  if (!ArmMmuEnabled () || !IsLiveBlockMapping) {
     *Entry = Value;
+    ArmUpdateTranslationTableEntry (Entry, (VOID *)(UINTN)RegionStart);
   } else {
     ArmReplaceLiveTranslationEntry (Entry, Value, RegionStart);
   }
@@ -136,258 +95,214 @@ ReplaceLiveEntry (
 
 STATIC
 VOID
-LookupAddresstoRootTable (
-  IN  UINT64  MaxAddress,
-  OUT UINTN  *T0SZ,
-  OUT UINTN  *TableEntryCount
+FreePageTablesRecursive (
+  IN  UINT64  *TranslationTable,
+  IN  UINTN   Level
   )
 {
-  UINTN TopBit;
+  UINTN   Index;
 
-  // Check the parameters are not NULL
-  ASSERT ((T0SZ != NULL) && (TableEntryCount != NULL));
+  ASSERT (Level <= 3);
 
-  // Look for the highest bit set in MaxAddress
-  for (TopBit = 63; TopBit != 0; TopBit--) {
-    if ((1ULL << TopBit) & MaxAddress) {
-      // MaxAddress top bit is found
-      TopBit = TopBit + 1;
-      break;
+  if (Level < 3) {
+    for (Index = 0; Index < TT_ENTRY_COUNT; Index++) {
+      if ((TranslationTable[Index] & TT_TYPE_MASK) == TT_TYPE_TABLE_ENTRY) {
+        FreePageTablesRecursive ((VOID *)(UINTN)(TranslationTable[Index] &
+                                                 TT_ADDRESS_MASK_BLOCK_ENTRY),
+                                 Level + 1);
+      }
     }
   }
-  ASSERT (TopBit != 0);
-
-  // Calculate T0SZ from the top bit of the MaxAddress
-  *T0SZ = 64 - TopBit;
-
-  // Get the Table info from T0SZ
-  GetRootTranslationTableInfo (*T0SZ, NULL, TableEntryCount);
+  FreePages (TranslationTable, 1);
 }
 
 STATIC
-UINT64*
-GetBlockEntryListFromAddress (
-  IN  UINT64       *RootTable,
-  IN  UINT64        RegionStart,
-  OUT UINTN        *TableLevel,
-  IN OUT UINT64    *BlockEntrySize,
-  OUT UINT64      **LastBlockEntry
+BOOLEAN
+IsBlockEntry (
+  IN  UINT64  Entry,
+  IN  UINTN   Level
   )
 {
-  UINTN   RootTableLevel;
-  UINTN   RootTableEntryCount;
-  UINT64 *TranslationTable;
-  UINT64 *BlockEntry;
-  UINT64 *SubTableBlockEntry;
-  UINT64  BlockEntryAddress;
-  UINTN   BaseAddressAlignment;
-  UINTN   PageLevel;
-  UINTN   Index;
-  UINTN   IndexLevel;
-  UINTN   T0SZ;
-  UINT64  Attributes;
-  UINT64  TableAttributes;
-
-  // Initialize variable
-  BlockEntry = NULL;
-
-  // Ensure the parameters are valid
-  if (!(TableLevel && BlockEntrySize && LastBlockEntry)) {
-    ASSERT_EFI_ERROR (EFI_INVALID_PARAMETER);
-    return NULL;
+  if (Level == 3) {
+    return (Entry & TT_TYPE_MASK) == TT_TYPE_BLOCK_ENTRY_LEVEL3;
   }
+  return (Entry & TT_TYPE_MASK) == TT_TYPE_BLOCK_ENTRY;
+}
 
-  // Ensure the Region is aligned on 4KB boundary
-  if ((RegionStart & (SIZE_4KB - 1)) != 0) {
-    ASSERT_EFI_ERROR (EFI_INVALID_PARAMETER);
-    return NULL;
+STATIC
+BOOLEAN
+IsTableEntry (
+  IN  UINT64  Entry,
+  IN  UINTN   Level
+  )
+{
+  if (Level == 3) {
+    //
+    // TT_TYPE_TABLE_ENTRY aliases TT_TYPE_BLOCK_ENTRY_LEVEL3
+    // so we need to take the level into account as well.
+    //
+    return FALSE;
   }
+  return (Entry & TT_TYPE_MASK) == TT_TYPE_TABLE_ENTRY;
+}
 
-  // Ensure the required size is aligned on 4KB boundary and not 0
-  if ((*BlockEntrySize & (SIZE_4KB - 1)) != 0 || *BlockEntrySize == 0) {
-    ASSERT_EFI_ERROR (EFI_INVALID_PARAMETER);
-    return NULL;
-  }
+STATIC
+EFI_STATUS
+UpdateRegionMappingRecursive (
+  IN  UINT64      RegionStart,
+  IN  UINT64      RegionEnd,
+  IN  UINT64      AttributeSetMask,
+  IN  UINT64      AttributeClearMask,
+  IN  UINT64      *PageTable,
+  IN  UINTN       Level
+  )
+{
+  UINTN           BlockShift;
+  UINT64          BlockMask;
+  UINT64          BlockEnd;
+  UINT64          *Entry;
+  UINT64          EntryValue;
+  VOID            *TranslationTable;
+  EFI_STATUS      Status;
 
-  T0SZ = ArmGetTCR () & TCR_T0SZ_MASK;
-  // Get the Table info from T0SZ
-  GetRootTranslationTableInfo (T0SZ, &RootTableLevel, &RootTableEntryCount);
+  ASSERT (((RegionStart | RegionEnd) & EFI_PAGE_MASK) == 0);
 
-  // If the start address is 0x0 then we use the size of the region to identify the alignment
-  if (RegionStart == 0) {
-    // Identify the highest possible alignment for the Region Size
-    BaseAddressAlignment = LowBitSet64 (*BlockEntrySize);
-  } else {
-    // Identify the highest possible alignment for the Base Address
-    BaseAddressAlignment = LowBitSet64 (RegionStart);
-  }
+  BlockShift = (Level + 1) * BITS_PER_LEVEL + MIN_T0SZ;
+  BlockMask = MAX_UINT64 >> BlockShift;
 
-  // Identify the Page Level the RegionStart must belong to. Note that PageLevel
-  // should be at least 1 since block translations are not supported at level 0
-  PageLevel = MAX (3 - ((BaseAddressAlignment - 12) / 9), 1);
+  DEBUG ((DEBUG_VERBOSE, "%a(%d): %llx - %llx set %lx clr %lx\n", __FUNCTION__,
+    Level, RegionStart, RegionEnd, AttributeSetMask, AttributeClearMask));
 
-  // If the required size is smaller than the current block size then we need to go to the page below.
-  // The PageLevel was calculated on the Base Address alignment but did not take in account the alignment
-  // of the allocation size
-  while (*BlockEntrySize < TT_BLOCK_ENTRY_SIZE_AT_LEVEL (PageLevel)) {
-    // It does not fit so we need to go a page level above
-    PageLevel++;
-  }
+  for (; RegionStart < RegionEnd; RegionStart = BlockEnd) {
+    BlockEnd = MIN (RegionEnd, (RegionStart | BlockMask) + 1);
+    Entry = &PageTable[(RegionStart >> (64 - BlockShift)) & (TT_ENTRY_COUNT - 1)];
 
-  //
-  // Get the Table Descriptor for the corresponding PageLevel. We need to decompose RegionStart to get appropriate entries
-  //
+    //
+    // If RegionStart or BlockEnd is not aligned to the block size at this
+    // level, we will have to create a table mapping in order to map less
+    // than a block, and recurse to create the block or page entries at
+    // the next level. No block mappings are allowed at all at level 0,
+    // so in that case, we have to recurse unconditionally.
+    // If we are changing a table entry and the AttributeClearMask is non-zero,
+    // we cannot replace it with a block entry without potentially losing
+    // attribute information, so keep the table entry in that case.
+    //
+    if (Level == 0 || ((RegionStart | BlockEnd) & BlockMask) != 0 ||
+        (IsTableEntry (*Entry, Level) && AttributeClearMask != 0)) {
+      ASSERT (Level < 3);
 
-  TranslationTable = RootTable;
-  for (IndexLevel = RootTableLevel; IndexLevel <= PageLevel; IndexLevel++) {
-    BlockEntry = (UINT64*)TT_GET_ENTRY_FOR_ADDRESS (TranslationTable, IndexLevel, RegionStart);
-
-    if ((IndexLevel != 3) && ((*BlockEntry & TT_TYPE_MASK) == TT_TYPE_TABLE_ENTRY)) {
-      // Go to the next table
-      TranslationTable = (UINT64*)(*BlockEntry & TT_ADDRESS_MASK_DESCRIPTION_TABLE);
-
-      // If we are at the last level then update the last level to next level
-      if (IndexLevel == PageLevel) {
-        // Enter the next level
-        PageLevel++;
-      }
-    } else if ((*BlockEntry & TT_TYPE_MASK) == TT_TYPE_BLOCK_ENTRY) {
-      // If we are not at the last level then we need to split this BlockEntry
-      if (IndexLevel != PageLevel) {
-        // Retrieve the attributes from the block entry
-        Attributes = *BlockEntry & TT_ATTRIBUTES_MASK;
-
-        // Convert the block entry attributes into Table descriptor attributes
-        TableAttributes = TT_TABLE_AP_NO_PERMISSION;
-        if (Attributes & TT_NS) {
-          TableAttributes = TT_TABLE_NS;
-        }
-
-        // Get the address corresponding at this entry
-        BlockEntryAddress = RegionStart;
-        BlockEntryAddress = BlockEntryAddress >> TT_ADDRESS_OFFSET_AT_LEVEL(IndexLevel);
-        // Shift back to right to set zero before the effective address
-        BlockEntryAddress = BlockEntryAddress << TT_ADDRESS_OFFSET_AT_LEVEL(IndexLevel);
-
-        // Set the correct entry type for the next page level
-        if ((IndexLevel + 1) == 3) {
-          Attributes |= TT_TYPE_BLOCK_ENTRY_LEVEL3;
-        } else {
-          Attributes |= TT_TYPE_BLOCK_ENTRY;
-        }
-
-        // Create a new translation table
+      if (!IsTableEntry (*Entry, Level)) {
+        //
+        // No table entry exists yet, so we need to allocate a page table
+        // for the next level.
+        //
         TranslationTable = AllocatePages (1);
         if (TranslationTable == NULL) {
-          return NULL;
+          return EFI_OUT_OF_RESOURCES;
         }
 
-        // Populate the newly created lower level table
-        SubTableBlockEntry = TranslationTable;
-        for (Index = 0; Index < TT_ENTRY_COUNT; Index++) {
-          *SubTableBlockEntry = Attributes | (BlockEntryAddress + (Index << TT_ADDRESS_OFFSET_AT_LEVEL(IndexLevel + 1)));
-          SubTableBlockEntry++;
+        if (!ArmMmuEnabled ()) {
+          //
+          // Make sure we are not inadvertently hitting in the caches
+          // when populating the page tables.
+          //
+          InvalidateDataCacheRange (TranslationTable, EFI_PAGE_SIZE);
         }
 
-        // Fill the BlockEntry with the new TranslationTable
-        ReplaceLiveEntry (BlockEntry,
-          (UINTN)TranslationTable | TableAttributes | TT_TYPE_TABLE_ENTRY,
-          RegionStart);
+        ZeroMem (TranslationTable, EFI_PAGE_SIZE);
+
+        if (IsBlockEntry (*Entry, Level)) {
+          //
+          // We are splitting an existing block entry, so we have to populate
+          // the new table with the attributes of the block entry it replaces.
+          //
+          Status = UpdateRegionMappingRecursive (RegionStart & ~BlockMask,
+                     (RegionStart | BlockMask) + 1, *Entry & TT_ATTRIBUTES_MASK,
+                     0, TranslationTable, Level + 1);
+          if (EFI_ERROR (Status)) {
+            //
+            // The range we passed to UpdateRegionMappingRecursive () is block
+            // aligned, so it is guaranteed that no further pages were allocated
+            // by it, and so we only have to free the page we allocated here.
+            //
+            FreePages (TranslationTable, 1);
+            return Status;
+          }
+        }
+      } else {
+        TranslationTable = (VOID *)(UINTN)(*Entry & TT_ADDRESS_MASK_BLOCK_ENTRY);
+      }
+
+      //
+      // Recurse to the next level
+      //
+      Status = UpdateRegionMappingRecursive (RegionStart, BlockEnd,
+                 AttributeSetMask, AttributeClearMask, TranslationTable,
+                 Level + 1);
+      if (EFI_ERROR (Status)) {
+        if (!IsTableEntry (*Entry, Level)) {
+          //
+          // We are creating a new table entry, so on failure, we can free all
+          // allocations we made recursively, given that the whole subhierarchy
+          // has not been wired into the live page tables yet. (This is not
+          // possible for existing table entries, since we cannot revert the
+          // modifications we made to the subhierarchy it represents.)
+          //
+          FreePageTablesRecursive (TranslationTable, Level + 1);
+        }
+        return Status;
+      }
+
+      if (!IsTableEntry (*Entry, Level)) {
+        EntryValue = (UINTN)TranslationTable | TT_TYPE_TABLE_ENTRY;
+        ReplaceTableEntry (Entry, EntryValue, RegionStart,
+          IsBlockEntry (*Entry, Level));
       }
     } else {
-      if (IndexLevel != PageLevel) {
+      EntryValue = (*Entry & AttributeClearMask) | AttributeSetMask;
+      EntryValue |= RegionStart;
+      EntryValue |= (Level == 3) ? TT_TYPE_BLOCK_ENTRY_LEVEL3
+                                 : TT_TYPE_BLOCK_ENTRY;
+
+      if (IsTableEntry (*Entry, Level)) {
         //
-        // Case when we have an Invalid Entry and we are at a page level above of the one targetted.
+        // We are replacing a table entry with a block entry. This is only
+        // possible if we are keeping none of the original attributes.
+        // We can free the table entry's page table, and all the ones below
+        // it, since we are dropping the only possible reference to it.
         //
-
-        // Create a new translation table
-        TranslationTable = AllocatePages (1);
-        if (TranslationTable == NULL) {
-          return NULL;
-        }
-
-        ZeroMem (TranslationTable, TT_ENTRY_COUNT * sizeof(UINT64));
-
-        // Fill the new BlockEntry with the TranslationTable
-        *BlockEntry = ((UINTN)TranslationTable & TT_ADDRESS_MASK_DESCRIPTION_TABLE) | TT_TYPE_TABLE_ENTRY;
+        ASSERT (AttributeClearMask == 0);
+        TranslationTable = (VOID *)(UINTN)(*Entry & TT_ADDRESS_MASK_BLOCK_ENTRY);
+        ReplaceTableEntry (Entry, EntryValue, RegionStart, TRUE);
+        FreePageTablesRecursive (TranslationTable, Level + 1);
+      } else {
+        ReplaceTableEntry (Entry, EntryValue, RegionStart, FALSE);
       }
     }
   }
-
-  // Expose the found PageLevel to the caller
-  *TableLevel = PageLevel;
-
-  // Now, we have the Table Level we can get the Block Size associated to this table
-  *BlockEntrySize = TT_BLOCK_ENTRY_SIZE_AT_LEVEL (PageLevel);
-
-  // The last block of the root table depends on the number of entry in this table,
-  // otherwise it is always the (TT_ENTRY_COUNT - 1)th entry in the table.
-  *LastBlockEntry = TT_LAST_BLOCK_ADDRESS(TranslationTable,
-      (PageLevel == RootTableLevel) ? RootTableEntryCount : TT_ENTRY_COUNT);
-
-  return BlockEntry;
+  return EFI_SUCCESS;
 }
 
 STATIC
 EFI_STATUS
 UpdateRegionMapping (
-  IN  UINT64  *RootTable,
   IN  UINT64  RegionStart,
   IN  UINT64  RegionLength,
-  IN  UINT64  Attributes,
-  IN  UINT64  BlockEntryMask
+  IN  UINT64  AttributeSetMask,
+  IN  UINT64  AttributeClearMask
   )
 {
-  UINT32  Type;
-  UINT64  *BlockEntry;
-  UINT64  *LastBlockEntry;
-  UINT64  BlockEntrySize;
-  UINTN   TableLevel;
+  UINTN     T0SZ;
 
-  // Ensure the Length is aligned on 4KB boundary
-  if ((RegionLength == 0) || ((RegionLength & (SIZE_4KB - 1)) != 0)) {
-    ASSERT_EFI_ERROR (EFI_INVALID_PARAMETER);
+  if (((RegionStart | RegionLength) & EFI_PAGE_MASK)) {
     return EFI_INVALID_PARAMETER;
   }
 
-  do {
-    // Get the first Block Entry that matches the Virtual Address and also the information on the Table Descriptor
-    // such as the size of the Block Entry and the address of the last BlockEntry of the Table Descriptor
-    BlockEntrySize = RegionLength;
-    BlockEntry = GetBlockEntryListFromAddress (RootTable, RegionStart, &TableLevel, &BlockEntrySize, &LastBlockEntry);
-    if (BlockEntry == NULL) {
-      // GetBlockEntryListFromAddress() return NULL when it fails to allocate new pages from the Translation Tables
-      return EFI_OUT_OF_RESOURCES;
-    }
+  T0SZ = ArmGetTCR () & TCR_T0SZ_MASK;
 
-    if (TableLevel != 3) {
-      Type = TT_TYPE_BLOCK_ENTRY;
-    } else {
-      Type = TT_TYPE_BLOCK_ENTRY_LEVEL3;
-    }
-
-    do {
-      // Fill the Block Entry with attribute and output block address
-      *BlockEntry &= BlockEntryMask;
-      *BlockEntry |= (RegionStart & TT_ADDRESS_MASK_BLOCK_ENTRY) | Attributes | Type;
-
-      ArmUpdateTranslationTableEntry (BlockEntry, (VOID *)RegionStart);
-
-      // Go to the next BlockEntry
-      RegionStart += BlockEntrySize;
-      RegionLength -= BlockEntrySize;
-      BlockEntry++;
-
-      // Break the inner loop when next block is a table
-      // Rerun GetBlockEntryListFromAddress to avoid page table memory leak
-      if (TableLevel != 3 && BlockEntry <= LastBlockEntry &&
-          (*BlockEntry & TT_TYPE_MASK) == TT_TYPE_TABLE_ENTRY) {
-            break;
-      }
-    } while ((RegionLength >= BlockEntrySize) && (BlockEntry <= LastBlockEntry));
-  } while (RegionLength != 0);
-
-  return EFI_SUCCESS;
+  return UpdateRegionMappingRecursive (RegionStart, RegionStart + RegionLength,
+           AttributeSetMask, AttributeClearMask, ArmGetTTBR0BaseAddress (),
+           GetRootTableLevel (T0SZ));
 }
 
 STATIC
@@ -398,7 +313,6 @@ FillTranslationTable (
   )
 {
   return UpdateRegionMapping (
-           RootTable,
            MemoryRegion->VirtualBase,
            MemoryRegion->Length,
            ArmMemoryAttributeToPageAttribute (MemoryRegion->Attributes) | TT_AF,
@@ -455,8 +369,6 @@ ArmSetMemoryAttributes (
   IN UINT64                    Attributes
   )
 {
-  EFI_STATUS                   Status;
-  UINT64                      *TranslationTable;
   UINT64                       PageAttributes;
   UINT64                       PageAttributeMask;
 
@@ -473,19 +385,8 @@ ArmSetMemoryAttributes (
                           TT_PXN_MASK | TT_XN_MASK);
   }
 
-  TranslationTable = ArmGetTTBR0BaseAddress ();
-
-  Status = UpdateRegionMapping (
-             TranslationTable,
-             BaseAddress,
-             Length,
-             PageAttributes,
-             PageAttributeMask);
-  if (EFI_ERROR (Status)) {
-    return Status;
-  }
-
-  return EFI_SUCCESS;
+  return UpdateRegionMapping (BaseAddress, Length, PageAttributes,
+           PageAttributeMask);
 }
 
 STATIC
@@ -497,17 +398,7 @@ SetMemoryRegionAttribute (
   IN  UINT64                    BlockEntryMask
   )
 {
-  EFI_STATUS                   Status;
-  UINT64                       *RootTable;
-
-  RootTable = ArmGetTTBR0BaseAddress ();
-
-  Status = UpdateRegionMapping (RootTable, BaseAddress, Length, Attributes, BlockEntryMask);
-  if (EFI_ERROR (Status)) {
-    return Status;
-  }
-
-  return EFI_SUCCESS;
+  return UpdateRegionMapping (BaseAddress, Length, Attributes, BlockEntryMask);
 }
 
 EFI_STATUS
@@ -584,14 +475,14 @@ ArmConfigureMmu (
   )
 {
   VOID*                         TranslationTable;
-  UINT32                        TranslationTableAttribute;
+  UINTN                         MaxAddressBits;
   UINT64                        MaxAddress;
   UINTN                         T0SZ;
   UINTN                         RootTableEntryCount;
   UINT64                        TCR;
   EFI_STATUS                    Status;
 
-  if(MemoryTable == NULL) {
+  if (MemoryTable == NULL) {
     ASSERT (MemoryTable != NULL);
     return EFI_INVALID_PARAMETER;
   }
@@ -603,11 +494,11 @@ ArmConfigureMmu (
   // into account the architectural limitations that result from UEFI's
   // use of 4 KB pages.
   //
-  MaxAddress = MIN (LShiftU64 (1ULL, ArmGetPhysicalAddressBits ()) - 1,
-                    MAX_ALLOC_ADDRESS);
+  MaxAddressBits = MIN (ArmGetPhysicalAddressBits (), MAX_VA_BITS);
+  MaxAddress = LShiftU64 (1ULL, MaxAddressBits) - 1;
 
-  // Lookup the Table Level to get the information
-  LookupAddresstoRootTable (MaxAddress, &T0SZ, &RootTableEntryCount);
+  T0SZ = 64 - MaxAddressBits;
+  RootTableEntryCount = GetRootTableEntryCount (T0SZ);
 
   //
   // Set TCR that allows us to retrieve T0SZ in the subsequent functions
@@ -632,7 +523,9 @@ ArmConfigureMmu (
     } else if (MaxAddress < SIZE_256TB) {
       TCR |= TCR_PS_256TB;
     } else {
-      DEBUG ((EFI_D_ERROR, "ArmConfigureMmu: The MaxAddress 0x%lX is not supported by this MMU configuration.\n", MaxAddress));
+      DEBUG ((DEBUG_ERROR,
+        "ArmConfigureMmu: The MaxAddress 0x%lX is not supported by this MMU configuration.\n",
+        MaxAddress));
       ASSERT (0); // Bigger than 48-bit memory space are not supported
       return EFI_UNSUPPORTED;
     }
@@ -654,7 +547,9 @@ ArmConfigureMmu (
     } else if (MaxAddress < SIZE_256TB) {
       TCR |= TCR_IPS_256TB;
     } else {
-      DEBUG ((EFI_D_ERROR, "ArmConfigureMmu: The MaxAddress 0x%lX is not supported by this MMU configuration.\n", MaxAddress));
+      DEBUG ((DEBUG_ERROR,
+        "ArmConfigureMmu: The MaxAddress 0x%lX is not supported by this MMU configuration.\n",
+        MaxAddress));
       ASSERT (0); // Bigger than 48-bit memory space are not supported
       return EFI_UNSUPPORTED;
     }
@@ -684,9 +579,12 @@ ArmConfigureMmu (
   if (TranslationTable == NULL) {
     return EFI_OUT_OF_RESOURCES;
   }
-  // We set TTBR0 just after allocating the table to retrieve its location from the subsequent
-  // functions without needing to pass this value across the functions. The MMU is only enabled
-  // after the translation tables are populated.
+  //
+  // We set TTBR0 just after allocating the table to retrieve its location from
+  // the subsequent functions without needing to pass this value across the
+  // functions. The MMU is only enabled after the translation tables are
+  // populated.
+  //
   ArmSetTTBR0 (TranslationTable);
 
   if (TranslationTableBase != NULL) {
@@ -694,46 +592,37 @@ ArmConfigureMmu (
   }
 
   if (TranslationTableSize != NULL) {
-    *TranslationTableSize = RootTableEntryCount * sizeof(UINT64);
+    *TranslationTableSize = RootTableEntryCount * sizeof (UINT64);
   }
 
-  ZeroMem (TranslationTable, RootTableEntryCount * sizeof(UINT64));
+  //
+  // Make sure we are not inadvertently hitting in the caches
+  // when populating the page tables.
+  //
+  InvalidateDataCacheRange (TranslationTable,
+    RootTableEntryCount * sizeof (UINT64));
+  ZeroMem (TranslationTable, RootTableEntryCount * sizeof (UINT64));
 
-  // Disable MMU and caches. ArmDisableMmu() also invalidates the TLBs
-  ArmDisableMmu ();
-  ArmDisableDataCache ();
-  ArmDisableInstructionCache ();
-
-  // Make sure nothing sneaked into the cache
-  ArmCleanInvalidateDataCache ();
-  ArmInvalidateInstructionCache ();
-
-  TranslationTableAttribute = TT_ATTR_INDX_INVALID;
   while (MemoryTable->Length != 0) {
-
-    DEBUG_CODE_BEGIN ();
-      // Find the memory attribute for the Translation Table
-      if ((UINTN)TranslationTable >= MemoryTable->PhysicalBase &&
-          (UINTN)TranslationTable + EFI_PAGE_SIZE <= MemoryTable->PhysicalBase +
-                                                          MemoryTable->Length) {
-        TranslationTableAttribute = MemoryTable->Attributes;
-      }
-    DEBUG_CODE_END ();
-
     Status = FillTranslationTable (TranslationTable, MemoryTable);
     if (EFI_ERROR (Status)) {
-      goto FREE_TRANSLATION_TABLE;
+      goto FreeTranslationTable;
     }
     MemoryTable++;
   }
 
-  ASSERT (TranslationTableAttribute == ARM_MEMORY_REGION_ATTRIBUTE_WRITE_BACK ||
-          TranslationTableAttribute == ARM_MEMORY_REGION_ATTRIBUTE_NONSECURE_WRITE_BACK);
-
-  ArmSetMAIR (MAIR_ATTR(TT_ATTR_INDX_DEVICE_MEMORY, MAIR_ATTR_DEVICE_MEMORY) |                      // mapped to EFI_MEMORY_UC
-              MAIR_ATTR(TT_ATTR_INDX_MEMORY_NON_CACHEABLE, MAIR_ATTR_NORMAL_MEMORY_NON_CACHEABLE) | // mapped to EFI_MEMORY_WC
-              MAIR_ATTR(TT_ATTR_INDX_MEMORY_WRITE_THROUGH, MAIR_ATTR_NORMAL_MEMORY_WRITE_THROUGH) | // mapped to EFI_MEMORY_WT
-              MAIR_ATTR(TT_ATTR_INDX_MEMORY_WRITE_BACK, MAIR_ATTR_NORMAL_MEMORY_WRITE_BACK));       // mapped to EFI_MEMORY_WB
+  //
+  // EFI_MEMORY_UC ==> MAIR_ATTR_DEVICE_MEMORY
+  // EFI_MEMORY_WC ==> MAIR_ATTR_NORMAL_MEMORY_NON_CACHEABLE
+  // EFI_MEMORY_WT ==> MAIR_ATTR_NORMAL_MEMORY_WRITE_THROUGH
+  // EFI_MEMORY_WB ==> MAIR_ATTR_NORMAL_MEMORY_WRITE_BACK
+  //
+  ArmSetMAIR (
+    MAIR_ATTR (TT_ATTR_INDX_DEVICE_MEMORY,        MAIR_ATTR_DEVICE_MEMORY)               |
+    MAIR_ATTR (TT_ATTR_INDX_MEMORY_NON_CACHEABLE, MAIR_ATTR_NORMAL_MEMORY_NON_CACHEABLE) |
+    MAIR_ATTR (TT_ATTR_INDX_MEMORY_WRITE_THROUGH, MAIR_ATTR_NORMAL_MEMORY_WRITE_THROUGH) |
+    MAIR_ATTR (TT_ATTR_INDX_MEMORY_WRITE_BACK,    MAIR_ATTR_NORMAL_MEMORY_WRITE_BACK)
+    );
 
   ArmDisableAlignmentCheck ();
   ArmEnableStackAlignmentCheck ();
@@ -743,7 +632,7 @@ ArmConfigureMmu (
   ArmEnableMmu ();
   return EFI_SUCCESS;
 
-FREE_TRANSLATION_TABLE:
+FreeTranslationTable:
   FreePages (TranslationTable, 1);
   return Status;
 }

ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibConvert.c

@@ -0,0 +1,32 @@
+/** @file
+*  File managing the MMU for ARMv7 architecture
+*
+*  Copyright (c) 2011-2016, ARM Limited. All rights reserved.
+*
+*  SPDX-License-Identifier: BSD-2-Clause-Patent
+*
+**/
+
+#include <Uefi.h>
+
+#include <Library/ArmLib.h>
+
+#include <Chipset/ArmV7.h>
+
+UINT32
+ConvertSectionAttributesToPageAttributes (
+  IN UINT32   SectionAttributes,
+  IN BOOLEAN  IsLargePage
+  )
+{
+  UINT32 PageAttributes;
+
+  PageAttributes = 0;
+  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_CACHE_POLICY (SectionAttributes, IsLargePage);
+  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_AP (SectionAttributes);
+  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_XN (SectionAttributes, IsLargePage);
+  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_NG (SectionAttributes);
+  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_S (SectionAttributes);
+
+  return PageAttributes;
+}

ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibCore.c

@@ -31,15 +31,6 @@
 #define ID_MMFR0_SHR_IMP_HW_COHERENT   1
 #define ID_MMFR0_SHR_IGNORED         0xf
 
-#define __EFI_MEMORY_RWX               0    // no restrictions
-
-#define CACHE_ATTRIBUTE_MASK   (EFI_MEMORY_UC | \
-                                EFI_MEMORY_WC | \
-                                EFI_MEMORY_WT | \
-                                EFI_MEMORY_WB | \
-                                EFI_MEMORY_UCE | \
-                                EFI_MEMORY_WP)
-
 UINTN
 EFIAPI
 ArmReadIdMmfr0 (
@@ -52,24 +43,6 @@ ArmHasMpExtensions (
   VOID
   );
 
-UINT32
-ConvertSectionAttributesToPageAttributes (
-  IN UINT32   SectionAttributes,
-  IN BOOLEAN  IsLargePage
-  )
-{
-  UINT32 PageAttributes;
-
-  PageAttributes = 0;
-  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_CACHE_POLICY (SectionAttributes, IsLargePage);
-  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_AP (SectionAttributes);
-  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_XN (SectionAttributes, IsLargePage);
-  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_NG (SectionAttributes);
-  PageAttributes |= TT_DESCRIPTOR_CONVERT_TO_PAGE_S (SectionAttributes);
-
-  return PageAttributes;
-}
-
 STATIC
 BOOLEAN
 PreferNonshareableMemory (
@@ -165,14 +138,22 @@ PopulateLevel2PageTable (
       // Case where a virtual memory map descriptor overlapped a section entry
 
       // Allocate a Level2 Page Table for this Section
-      TranslationTable = (UINTN)AllocatePages(EFI_SIZE_TO_PAGES(TRANSLATION_TABLE_PAGE_SIZE + TRANSLATION_TABLE_PAGE_ALIGNMENT));
-      TranslationTable = ((UINTN)TranslationTable + TRANSLATION_TABLE_PAGE_ALIGNMENT_MASK) & ~TRANSLATION_TABLE_PAGE_ALIGNMENT_MASK;
+      TranslationTable = (UINTN)AllocateAlignedPages (
+                                  EFI_SIZE_TO_PAGES (TRANSLATION_TABLE_PAGE_SIZE),
+                                  TRANSLATION_TABLE_PAGE_ALIGNMENT);
 
       // Translate the Section Descriptor into Page Descriptor
       SectionDescriptor = TT_DESCRIPTOR_PAGE_TYPE_PAGE | ConvertSectionAttributesToPageAttributes (*SectionEntry, FALSE);
 
       BaseSectionAddress = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(*SectionEntry);
 
+      //
+      // Make sure we are not inadvertently hitting in the caches
+      // when populating the page tables
+      //
+      InvalidateDataCacheRange ((VOID *)TranslationTable,
+        TRANSLATION_TABLE_PAGE_SIZE);
+
       // Populate the new Level2 Page Table for the section
       PageEntry = (UINT32*)TranslationTable;
       for (Index = 0; Index < TRANSLATION_TABLE_PAGE_COUNT; Index++) {
@@ -189,9 +170,15 @@ PopulateLevel2PageTable (
       return;
     }
   } else {
-    TranslationTable = (UINTN)AllocatePages(EFI_SIZE_TO_PAGES(TRANSLATION_TABLE_PAGE_SIZE + TRANSLATION_TABLE_PAGE_ALIGNMENT));
-    TranslationTable = ((UINTN)TranslationTable + TRANSLATION_TABLE_PAGE_ALIGNMENT_MASK) & ~TRANSLATION_TABLE_PAGE_ALIGNMENT_MASK;
-
+    TranslationTable = (UINTN)AllocateAlignedPages (
+                                EFI_SIZE_TO_PAGES (TRANSLATION_TABLE_PAGE_SIZE),
+                                TRANSLATION_TABLE_PAGE_ALIGNMENT);
+    //
+    // Make sure we are not inadvertently hitting in the caches
+    // when populating the page tables
+    //
+    InvalidateDataCacheRange ((VOID *)TranslationTable,
+      TRANSLATION_TABLE_PAGE_SIZE);
     ZeroMem ((VOID *)TranslationTable, TRANSLATION_TABLE_PAGE_SIZE);
 
     *SectionEntry = (TranslationTable & TT_DESCRIPTOR_SECTION_PAGETABLE_ADDRESS_MASK) |
@@ -210,6 +197,13 @@ PopulateLevel2PageTable (
     PhysicalBase += TT_DESCRIPTOR_PAGE_SIZE;
   }
 
+  //
+  // Invalidate again to ensure that any line fetches that may have occurred
+  // [speculatively] since the previous invalidate are evicted again.
+  //
+  ArmDataMemoryBarrier ();
+  InvalidateDataCacheRange ((UINT32 *)TranslationTable + FirstPageOffset,
+    RemainLength / TT_DESCRIPTOR_PAGE_SIZE * sizeof (*PageEntry));
 }
 
 STATIC
@@ -284,7 +278,16 @@ FillTranslationTable (
         RemainLength >= TT_DESCRIPTOR_SECTION_SIZE) {
       // Case: Physical address aligned on the Section Size (1MB) && the length
       // is greater than the Section Size
-      *SectionEntry++ = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(PhysicalBase) | Attributes;
+      *SectionEntry = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(PhysicalBase) | Attributes;
+
+      //
+      // Issue a DMB to ensure that the page table entry update made it to
+      // memory before we issue the invalidate, otherwise, a subsequent
+      // speculative fetch could observe the old value.
+      //
+      ArmDataMemoryBarrier ();
+      ArmInvalidateDataCacheEntryByMVA ((UINTN)SectionEntry++);
+
       PhysicalBase += TT_DESCRIPTOR_SECTION_SIZE;
       RemainLength -= TT_DESCRIPTOR_SECTION_SIZE;
     } else {
@@ -294,9 +297,17 @@ FillTranslationTable (
       // Case: Physical address aligned on the Section Size (1MB) && the length
       //       does not fill a section
       // Case: Physical address NOT aligned on the Section Size (1MB)
-      PopulateLevel2PageTable (SectionEntry++, PhysicalBase, PageMapLength,
+      PopulateLevel2PageTable (SectionEntry, PhysicalBase, PageMapLength,
         MemoryRegion->Attributes);
 
+      //
+      // Issue a DMB to ensure that the page table entry update made it to
+      // memory before we issue the invalidate, otherwise, a subsequent
+      // speculative fetch could observe the old value.
+      //
+      ArmDataMemoryBarrier ();
+      ArmInvalidateDataCacheEntryByMVA ((UINTN)SectionEntry++);
+
       // If it is the last entry
       if (RemainLength < TT_DESCRIPTOR_SECTION_SIZE) {
         break;
@@ -316,16 +327,15 @@ ArmConfigureMmu (
   OUT UINTN                         *TranslationTableSize OPTIONAL
   )
 {
-  VOID*                         TranslationTable;
-  ARM_MEMORY_REGION_ATTRIBUTES  TranslationTableAttribute;
+  VOID                          *TranslationTable;
   UINT32                        TTBRAttributes;
 
-  // Allocate pages for translation table.
-  TranslationTable = AllocatePages (EFI_SIZE_TO_PAGES (TRANSLATION_TABLE_SECTION_SIZE + TRANSLATION_TABLE_SECTION_ALIGNMENT));
+  TranslationTable = AllocateAlignedPages (
+                       EFI_SIZE_TO_PAGES (TRANSLATION_TABLE_SECTION_SIZE),
+                       TRANSLATION_TABLE_SECTION_ALIGNMENT);
   if (TranslationTable == NULL) {
     return RETURN_OUT_OF_RESOURCES;
   }
-  TranslationTable = (VOID*)(((UINTN)TranslationTable + TRANSLATION_TABLE_SECTION_ALIGNMENT_MASK) & ~TRANSLATION_TABLE_SECTION_ALIGNMENT_MASK);
 
   if (TranslationTableBase != NULL) {
     *TranslationTableBase = TranslationTable;
@@ -335,30 +345,20 @@ ArmConfigureMmu (
     *TranslationTableSize = TRANSLATION_TABLE_SECTION_SIZE;
   }
 
+  //
+  // Make sure we are not inadvertently hitting in the caches
+  // when populating the page tables
+  //
+  InvalidateDataCacheRange (TranslationTable, TRANSLATION_TABLE_SECTION_SIZE);
   ZeroMem (TranslationTable, TRANSLATION_TABLE_SECTION_SIZE);
 
-  // By default, mark the translation table as belonging to a uncached region
-  TranslationTableAttribute = ARM_MEMORY_REGION_ATTRIBUTE_UNCACHED_UNBUFFERED;
   while (MemoryTable->Length != 0) {
-    // Find the memory attribute for the Translation Table
-    if (((UINTN)TranslationTable >= MemoryTable->PhysicalBase) && ((UINTN)TranslationTable <= MemoryTable->PhysicalBase - 1 + MemoryTable->Length)) {
-      TranslationTableAttribute = MemoryTable->Attributes;
-    }
-
     FillTranslationTable (TranslationTable, MemoryTable);
     MemoryTable++;
   }
 
-  // Translate the Memory Attributes into Translation Table Register Attributes
-  if ((TranslationTableAttribute == ARM_MEMORY_REGION_ATTRIBUTE_WRITE_BACK) ||
-      (TranslationTableAttribute == ARM_MEMORY_REGION_ATTRIBUTE_NONSECURE_WRITE_BACK)) {
-    TTBRAttributes = ArmHasMpExtensions () ? TTBR_MP_WRITE_BACK_ALLOC : TTBR_WRITE_BACK_ALLOC;
-  } else {
-    // Page tables must reside in memory mapped as write-back cacheable
-    ASSERT (0);
-    return RETURN_UNSUPPORTED;
-  }
-
+  TTBRAttributes = ArmHasMpExtensions () ? TTBR_MP_WRITE_BACK_ALLOC
+                                         : TTBR_WRITE_BACK_ALLOC;
   if (TTBRAttributes & TTBR_SHAREABLE) {
     if (PreferNonshareableMemory ()) {
       TTBRAttributes ^= TTBR_SHAREABLE;
@@ -376,19 +376,7 @@ ArmConfigureMmu (
     }
   }
 
-  ArmCleanInvalidateDataCache ();
-  ArmInvalidateInstructionCache ();
-
-  ArmDisableDataCache ();
-  ArmDisableInstructionCache();
-  // TLBs are also invalidated when calling ArmDisableMmu()
-  ArmDisableMmu ();
-
-  // Make sure nothing sneaked into the cache
-  ArmCleanInvalidateDataCache ();
-  ArmInvalidateInstructionCache ();
-
-  ArmSetTTBR0 ((VOID *)(UINTN)(((UINTN)TranslationTable & ~TRANSLATION_TABLE_SECTION_ALIGNMENT_MASK) | (TTBRAttributes & 0x7F)));
+  ArmSetTTBR0 ((VOID *)((UINTN)TranslationTable | TTBRAttributes));
 
   //
   // The TTBCR register value is undefined at reset in the Non-Secure world.
@@ -423,419 +411,3 @@ ArmConfigureMmu (
   ArmEnableMmu();
   return RETURN_SUCCESS;
 }
-
-STATIC
-EFI_STATUS
-ConvertSectionToPages (
-  IN EFI_PHYSICAL_ADDRESS  BaseAddress
-  )
-{
-  UINT32                  FirstLevelIdx;
-  UINT32                  SectionDescriptor;
-  UINT32                  PageTableDescriptor;
-  UINT32                  PageDescriptor;
-  UINT32                  Index;
-
-  volatile ARM_FIRST_LEVEL_DESCRIPTOR   *FirstLevelTable;
-  volatile ARM_PAGE_TABLE_ENTRY         *PageTable;
-
-  DEBUG ((EFI_D_PAGE, "Converting section at 0x%x to pages\n", (UINTN)BaseAddress));
-
-  // Obtain page table base
-  FirstLevelTable = (ARM_FIRST_LEVEL_DESCRIPTOR *)ArmGetTTBR0BaseAddress ();
-
-  // Calculate index into first level translation table for start of modification
-  FirstLevelIdx = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(BaseAddress) >> TT_DESCRIPTOR_SECTION_BASE_SHIFT;
-  ASSERT (FirstLevelIdx < TRANSLATION_TABLE_SECTION_COUNT);
-
-  // Get section attributes and convert to page attributes
-  SectionDescriptor = FirstLevelTable[FirstLevelIdx];
-  PageDescriptor = TT_DESCRIPTOR_PAGE_TYPE_PAGE | ConvertSectionAttributesToPageAttributes (SectionDescriptor, FALSE);
-
-  // Allocate a page table for the 4KB entries (we use up a full page even though we only need 1KB)
-  PageTable = (volatile ARM_PAGE_TABLE_ENTRY *)AllocatePages (1);
-  if (PageTable == NULL) {
-    return EFI_OUT_OF_RESOURCES;
-  }
-
-  // Write the page table entries out
-  for (Index = 0; Index < TRANSLATION_TABLE_PAGE_COUNT; Index++) {
-    PageTable[Index] = TT_DESCRIPTOR_PAGE_BASE_ADDRESS(BaseAddress + (Index << 12)) | PageDescriptor;
-  }
-
-  // Formulate page table entry, Domain=0, NS=0
-  PageTableDescriptor = (((UINTN)PageTable) & TT_DESCRIPTOR_SECTION_PAGETABLE_ADDRESS_MASK) | TT_DESCRIPTOR_SECTION_TYPE_PAGE_TABLE;
-
-  // Write the page table entry out, replacing section entry
-  FirstLevelTable[FirstLevelIdx] = PageTableDescriptor;
-
-  return EFI_SUCCESS;
-}
-
-STATIC
-EFI_STATUS
-UpdatePageEntries (
-  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN  UINT64                    Length,
-  IN  UINT64                    Attributes,
-  OUT BOOLEAN                   *FlushTlbs OPTIONAL
-  )
-{
-  EFI_STATUS    Status;
-  UINT32        EntryValue;
-  UINT32        EntryMask;
-  UINT32        FirstLevelIdx;
-  UINT32        Offset;
-  UINT32        NumPageEntries;
-  UINT32        Descriptor;
-  UINT32        p;
-  UINT32        PageTableIndex;
-  UINT32        PageTableEntry;
-  UINT32        CurrentPageTableEntry;
-  VOID          *Mva;
-
-  volatile ARM_FIRST_LEVEL_DESCRIPTOR   *FirstLevelTable;
-  volatile ARM_PAGE_TABLE_ENTRY         *PageTable;
-
-  Status = EFI_SUCCESS;
-
-  // EntryMask: bitmask of values to change (1 = change this value, 0 = leave alone)
-  // EntryValue: values at bit positions specified by EntryMask
-  EntryMask = TT_DESCRIPTOR_PAGE_TYPE_MASK | TT_DESCRIPTOR_PAGE_AP_MASK;
-  if (Attributes & EFI_MEMORY_XP) {
-    EntryValue = TT_DESCRIPTOR_PAGE_TYPE_PAGE_XN;
-  } else {
-    EntryValue = TT_DESCRIPTOR_PAGE_TYPE_PAGE;
-  }
-
-  // Although the PI spec is unclear on this, the GCD guarantees that only
-  // one Attribute bit is set at a time, so the order of the conditionals below
-  // is irrelevant. If no memory attribute is specified, we preserve whatever
-  // memory type is set in the page tables, and update the permission attributes
-  // only.
-  if (Attributes & EFI_MEMORY_UC) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
-    // map to strongly ordered
-    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_STRONGLY_ORDERED; // TEX[2:0] = 0, C=0, B=0
-  } else if (Attributes & EFI_MEMORY_WC) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
-    // map to normal non-cachable
-    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_NON_CACHEABLE; // TEX [2:0]= 001 = 0x2, B=0, C=0
-  } else if (Attributes & EFI_MEMORY_WT) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
-    // write through with no-allocate
-    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_WRITE_THROUGH_NO_ALLOC; // TEX [2:0] = 0, C=1, B=0
-  } else if (Attributes & EFI_MEMORY_WB) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
-    // write back (with allocate)
-    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_WRITE_BACK_ALLOC; // TEX [2:0] = 001, C=1, B=1
-  } else if (Attributes & CACHE_ATTRIBUTE_MASK) {
-    // catch unsupported memory type attributes
-    ASSERT (FALSE);
-    return EFI_UNSUPPORTED;
-  }
-
-  if (Attributes & EFI_MEMORY_RO) {
-    EntryValue |= TT_DESCRIPTOR_PAGE_AP_RO_RO;
-  } else {
-    EntryValue |= TT_DESCRIPTOR_PAGE_AP_RW_RW;
-  }
-
-  // Obtain page table base
-  FirstLevelTable = (ARM_FIRST_LEVEL_DESCRIPTOR *)ArmGetTTBR0BaseAddress ();
-
-  // Calculate number of 4KB page table entries to change
-  NumPageEntries = Length / TT_DESCRIPTOR_PAGE_SIZE;
-
-  // Iterate for the number of 4KB pages to change
-  Offset = 0;
-  for(p = 0; p < NumPageEntries; p++) {
-    // Calculate index into first level translation table for page table value
-
-    FirstLevelIdx = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(BaseAddress + Offset) >> TT_DESCRIPTOR_SECTION_BASE_SHIFT;
-    ASSERT (FirstLevelIdx < TRANSLATION_TABLE_SECTION_COUNT);
-
-    // Read the descriptor from the first level page table
-    Descriptor = FirstLevelTable[FirstLevelIdx];
-
-    // Does this descriptor need to be converted from section entry to 4K pages?
-    if (!TT_DESCRIPTOR_SECTION_TYPE_IS_PAGE_TABLE(Descriptor)) {
-      Status = ConvertSectionToPages (FirstLevelIdx << TT_DESCRIPTOR_SECTION_BASE_SHIFT);
-      if (EFI_ERROR(Status)) {
-        // Exit for loop
-        break;
-      }
-
-      // Re-read descriptor
-      Descriptor = FirstLevelTable[FirstLevelIdx];
-      if (FlushTlbs != NULL) {
-        *FlushTlbs = TRUE;
-      }
-    }
-
-    // Obtain page table base address
-    PageTable = (ARM_PAGE_TABLE_ENTRY *)TT_DESCRIPTOR_PAGE_BASE_ADDRESS(Descriptor);
-
-    // Calculate index into the page table
-    PageTableIndex = ((BaseAddress + Offset) & TT_DESCRIPTOR_PAGE_INDEX_MASK) >> TT_DESCRIPTOR_PAGE_BASE_SHIFT;
-    ASSERT (PageTableIndex < TRANSLATION_TABLE_PAGE_COUNT);
-
-    // Get the entry
-    CurrentPageTableEntry = PageTable[PageTableIndex];
-
-    // Mask off appropriate fields
-    PageTableEntry = CurrentPageTableEntry & ~EntryMask;
-
-    // Mask in new attributes and/or permissions
-    PageTableEntry |= EntryValue;
-
-    if (CurrentPageTableEntry  != PageTableEntry) {
-      Mva = (VOID *)(UINTN)((((UINTN)FirstLevelIdx) << TT_DESCRIPTOR_SECTION_BASE_SHIFT) + (PageTableIndex << TT_DESCRIPTOR_PAGE_BASE_SHIFT));
-
-      // Only need to update if we are changing the entry
-      PageTable[PageTableIndex] = PageTableEntry;
-      ArmUpdateTranslationTableEntry ((VOID *)&PageTable[PageTableIndex], Mva);
-    }
-
-    Status = EFI_SUCCESS;
-    Offset += TT_DESCRIPTOR_PAGE_SIZE;
-
-  } // End first level translation table loop
-
-  return Status;
-}
-
-STATIC
-EFI_STATUS
-UpdateSectionEntries (
-  IN EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN UINT64                    Length,
-  IN UINT64                    Attributes
-  )
-{
-  EFI_STATUS    Status = EFI_SUCCESS;
-  UINT32        EntryMask;
-  UINT32        EntryValue;
-  UINT32        FirstLevelIdx;
-  UINT32        NumSections;
-  UINT32        i;
-  UINT32        CurrentDescriptor;
-  UINT32        Descriptor;
-  VOID          *Mva;
-  volatile ARM_FIRST_LEVEL_DESCRIPTOR   *FirstLevelTable;
-
-  // EntryMask: bitmask of values to change (1 = change this value, 0 = leave alone)
-  // EntryValue: values at bit positions specified by EntryMask
-
-  // Make sure we handle a section range that is unmapped
-  EntryMask = TT_DESCRIPTOR_SECTION_TYPE_MASK | TT_DESCRIPTOR_SECTION_XN_MASK |
-              TT_DESCRIPTOR_SECTION_AP_MASK;
-  EntryValue = TT_DESCRIPTOR_SECTION_TYPE_SECTION;
-
-  // Although the PI spec is unclear on this, the GCD guarantees that only
-  // one Attribute bit is set at a time, so the order of the conditionals below
-  // is irrelevant. If no memory attribute is specified, we preserve whatever
-  // memory type is set in the page tables, and update the permission attributes
-  // only.
-  if (Attributes & EFI_MEMORY_UC) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
-    // map to strongly ordered
-    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_STRONGLY_ORDERED; // TEX[2:0] = 0, C=0, B=0
-  } else if (Attributes & EFI_MEMORY_WC) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
-    // map to normal non-cachable
-    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_NON_CACHEABLE; // TEX [2:0]= 001 = 0x2, B=0, C=0
-  } else if (Attributes & EFI_MEMORY_WT) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
-    // write through with no-allocate
-    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_WRITE_THROUGH_NO_ALLOC; // TEX [2:0] = 0, C=1, B=0
-  } else if (Attributes & EFI_MEMORY_WB) {
-    // modify cacheability attributes
-    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
-    // write back (with allocate)
-    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_WRITE_BACK_ALLOC; // TEX [2:0] = 001, C=1, B=1
-  } else if (Attributes & CACHE_ATTRIBUTE_MASK) {
-    // catch unsupported memory type attributes
-    ASSERT (FALSE);
-    return EFI_UNSUPPORTED;
-  }
-
-  if (Attributes & EFI_MEMORY_RO) {
-    EntryValue |= TT_DESCRIPTOR_SECTION_AP_RO_RO;
-  } else {
-    EntryValue |= TT_DESCRIPTOR_SECTION_AP_RW_RW;
-  }
-
-  if (Attributes & EFI_MEMORY_XP) {
-    EntryValue |= TT_DESCRIPTOR_SECTION_XN_MASK;
-  }
-
-  // obtain page table base
-  FirstLevelTable = (ARM_FIRST_LEVEL_DESCRIPTOR *)ArmGetTTBR0BaseAddress ();
-
-  // calculate index into first level translation table for start of modification
-  FirstLevelIdx = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(BaseAddress) >> TT_DESCRIPTOR_SECTION_BASE_SHIFT;
-  ASSERT (FirstLevelIdx < TRANSLATION_TABLE_SECTION_COUNT);
-
-  // calculate number of 1MB first level entries this applies to
-  NumSections = Length / TT_DESCRIPTOR_SECTION_SIZE;
-
-  // iterate through each descriptor
-  for(i=0; i<NumSections; i++) {
-    CurrentDescriptor = FirstLevelTable[FirstLevelIdx + i];
-
-    // has this descriptor already been converted to pages?
-    if (TT_DESCRIPTOR_SECTION_TYPE_IS_PAGE_TABLE(CurrentDescriptor)) {
-      // forward this 1MB range to page table function instead
-      Status = UpdatePageEntries (
-                 (FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT,
-                 TT_DESCRIPTOR_SECTION_SIZE,
-                 Attributes,
-                 NULL);
-    } else {
-      // still a section entry
-
-      if (CurrentDescriptor != 0) {
-        // mask off appropriate fields
-        Descriptor = CurrentDescriptor & ~EntryMask;
-      } else {
-        Descriptor = ((UINTN)FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT;
-      }
-
-      // mask in new attributes and/or permissions
-      Descriptor |= EntryValue;
-
-      if (CurrentDescriptor  != Descriptor) {
-        Mva = (VOID *)(UINTN)(((UINTN)FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT);
-
-        // Only need to update if we are changing the descriptor
-        FirstLevelTable[FirstLevelIdx + i] = Descriptor;
-        ArmUpdateTranslationTableEntry ((VOID *)&FirstLevelTable[FirstLevelIdx + i], Mva);
-      }
-
-      Status = EFI_SUCCESS;
-    }
-  }
-
-  return Status;
-}
-
-EFI_STATUS
-ArmSetMemoryAttributes (
-  IN EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN UINT64                    Length,
-  IN UINT64                    Attributes
-  )
-{
-  EFI_STATUS    Status;
-  UINT64        ChunkLength;
-  BOOLEAN       FlushTlbs;
-
-  if (BaseAddress > (UINT64)MAX_ADDRESS) {
-    return EFI_UNSUPPORTED;
-  }
-
-  Length = MIN (Length, (UINT64)MAX_ADDRESS - BaseAddress + 1);
-  if (Length == 0) {
-    return EFI_SUCCESS;
-  }
-
-  FlushTlbs = FALSE;
-  while (Length > 0) {
-    if ((BaseAddress % TT_DESCRIPTOR_SECTION_SIZE == 0) &&
-        Length >= TT_DESCRIPTOR_SECTION_SIZE) {
-
-      ChunkLength = Length - Length % TT_DESCRIPTOR_SECTION_SIZE;
-
-      DEBUG ((DEBUG_PAGE,
-        "SetMemoryAttributes(): MMU section 0x%lx length 0x%lx to %lx\n",
-        BaseAddress, ChunkLength, Attributes));
-
-      Status = UpdateSectionEntries (BaseAddress, ChunkLength, Attributes);
-
-      FlushTlbs = TRUE;
-    } else {
-
-      //
-      // Process page by page until the next section boundary, but only if
-      // we have more than a section's worth of area to deal with after that.
-      //
-      ChunkLength = TT_DESCRIPTOR_SECTION_SIZE -
-                    (BaseAddress % TT_DESCRIPTOR_SECTION_SIZE);
-      if (ChunkLength + TT_DESCRIPTOR_SECTION_SIZE > Length) {
-        ChunkLength = Length;
-      }
-
-      DEBUG ((DEBUG_PAGE,
-        "SetMemoryAttributes(): MMU page 0x%lx length 0x%lx to %lx\n",
-        BaseAddress, ChunkLength, Attributes));
-
-      Status = UpdatePageEntries (BaseAddress, ChunkLength, Attributes,
-                 &FlushTlbs);
-    }
-
-    if (EFI_ERROR (Status)) {
-      break;
-    }
-
-    BaseAddress += ChunkLength;
-    Length -= ChunkLength;
-  }
-
-  if (FlushTlbs) {
-    ArmInvalidateTlb ();
-  }
-  return Status;
-}
-
-EFI_STATUS
-ArmSetMemoryRegionNoExec (
-  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN  UINT64                    Length
-  )
-{
-  return ArmSetMemoryAttributes (BaseAddress, Length, EFI_MEMORY_XP);
-}
-
-EFI_STATUS
-ArmClearMemoryRegionNoExec (
-  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN  UINT64                    Length
-  )
-{
-  return ArmSetMemoryAttributes (BaseAddress, Length, __EFI_MEMORY_RWX);
-}
-
-EFI_STATUS
-ArmSetMemoryRegionReadOnly (
-  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN  UINT64                    Length
-  )
-{
-  return ArmSetMemoryAttributes (BaseAddress, Length, EFI_MEMORY_RO);
-}
-
-EFI_STATUS
-ArmClearMemoryRegionReadOnly (
-  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
-  IN  UINT64                    Length
-  )
-{
-  return ArmSetMemoryAttributes (BaseAddress, Length, __EFI_MEMORY_RWX);
-}
-
-RETURN_STATUS
-EFIAPI
-ArmMmuBaseLibConstructor (
-  VOID
-  )
-{
-  return RETURN_SUCCESS;
-}

ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibUpdate.c

@@ -0,0 +1,435 @@
+/** @file
+*  File managing the MMU for ARMv7 architecture
+*
+*  Copyright (c) 2011-2016, ARM Limited. All rights reserved.
+*
+*  SPDX-License-Identifier: BSD-2-Clause-Patent
+*
+**/
+
+#include <Uefi.h>
+
+#include <Library/ArmLib.h>
+#include <Library/BaseLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Library/DebugLib.h>
+#include <Library/CacheMaintenanceLib.h>
+#include <Library/MemoryAllocationLib.h>
+
+#include <Chipset/ArmV7.h>
+
+#define __EFI_MEMORY_RWX               0    // no restrictions
+
+#define CACHE_ATTRIBUTE_MASK   (EFI_MEMORY_UC | \
+                                EFI_MEMORY_WC | \
+                                EFI_MEMORY_WT | \
+                                EFI_MEMORY_WB | \
+                                EFI_MEMORY_UCE | \
+                                EFI_MEMORY_WP)
+
+STATIC
+EFI_STATUS
+ConvertSectionToPages (
+  IN EFI_PHYSICAL_ADDRESS  BaseAddress
+  )
+{
+  UINT32                  FirstLevelIdx;
+  UINT32                  SectionDescriptor;
+  UINT32                  PageTableDescriptor;
+  UINT32                  PageDescriptor;
+  UINT32                  Index;
+
+  volatile ARM_FIRST_LEVEL_DESCRIPTOR   *FirstLevelTable;
+  volatile ARM_PAGE_TABLE_ENTRY         *PageTable;
+
+  DEBUG ((DEBUG_PAGE, "Converting section at 0x%x to pages\n", (UINTN)BaseAddress));
+
+  // Obtain page table base
+  FirstLevelTable = (ARM_FIRST_LEVEL_DESCRIPTOR *)ArmGetTTBR0BaseAddress ();
+
+  // Calculate index into first level translation table for start of modification
+  FirstLevelIdx = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(BaseAddress) >> TT_DESCRIPTOR_SECTION_BASE_SHIFT;
+  ASSERT (FirstLevelIdx < TRANSLATION_TABLE_SECTION_COUNT);
+
+  // Get section attributes and convert to page attributes
+  SectionDescriptor = FirstLevelTable[FirstLevelIdx];
+  PageDescriptor = TT_DESCRIPTOR_PAGE_TYPE_PAGE | ConvertSectionAttributesToPageAttributes (SectionDescriptor, FALSE);
+
+  // Allocate a page table for the 4KB entries (we use up a full page even though we only need 1KB)
+  PageTable = (volatile ARM_PAGE_TABLE_ENTRY *)AllocatePages (1);
+  if (PageTable == NULL) {
+    return EFI_OUT_OF_RESOURCES;
+  }
+
+  // Write the page table entries out
+  for (Index = 0; Index < TRANSLATION_TABLE_PAGE_COUNT; Index++) {
+    PageTable[Index] = TT_DESCRIPTOR_PAGE_BASE_ADDRESS(BaseAddress + (Index << 12)) | PageDescriptor;
+  }
+
+  // Formulate page table entry, Domain=0, NS=0
+  PageTableDescriptor = (((UINTN)PageTable) & TT_DESCRIPTOR_SECTION_PAGETABLE_ADDRESS_MASK) | TT_DESCRIPTOR_SECTION_TYPE_PAGE_TABLE;
+
+  // Write the page table entry out, replacing section entry
+  FirstLevelTable[FirstLevelIdx] = PageTableDescriptor;
+
+  return EFI_SUCCESS;
+}
+
+STATIC
+EFI_STATUS
+UpdatePageEntries (
+  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN  UINT64                    Length,
+  IN  UINT64                    Attributes,
+  OUT BOOLEAN                   *FlushTlbs OPTIONAL
+  )
+{
+  EFI_STATUS    Status;
+  UINT32        EntryValue;
+  UINT32        EntryMask;
+  UINT32        FirstLevelIdx;
+  UINT32        Offset;
+  UINT32        NumPageEntries;
+  UINT32        Descriptor;
+  UINT32        p;
+  UINT32        PageTableIndex;
+  UINT32        PageTableEntry;
+  UINT32        CurrentPageTableEntry;
+  VOID          *Mva;
+
+  volatile ARM_FIRST_LEVEL_DESCRIPTOR   *FirstLevelTable;
+  volatile ARM_PAGE_TABLE_ENTRY         *PageTable;
+
+  Status = EFI_SUCCESS;
+
+  // EntryMask: bitmask of values to change (1 = change this value, 0 = leave alone)
+  // EntryValue: values at bit positions specified by EntryMask
+  EntryMask = TT_DESCRIPTOR_PAGE_TYPE_MASK | TT_DESCRIPTOR_PAGE_AP_MASK;
+  if (Attributes & EFI_MEMORY_XP) {
+    EntryValue = TT_DESCRIPTOR_PAGE_TYPE_PAGE_XN;
+  } else {
+    EntryValue = TT_DESCRIPTOR_PAGE_TYPE_PAGE;
+  }
+
+  // Although the PI spec is unclear on this, the GCD guarantees that only
+  // one Attribute bit is set at a time, so the order of the conditionals below
+  // is irrelevant. If no memory attribute is specified, we preserve whatever
+  // memory type is set in the page tables, and update the permission attributes
+  // only.
+  if (Attributes & EFI_MEMORY_UC) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
+    // map to strongly ordered
+    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_STRONGLY_ORDERED; // TEX[2:0] = 0, C=0, B=0
+  } else if (Attributes & EFI_MEMORY_WC) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
+    // map to normal non-cachable
+    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_NON_CACHEABLE; // TEX [2:0]= 001 = 0x2, B=0, C=0
+  } else if (Attributes & EFI_MEMORY_WT) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
+    // write through with no-allocate
+    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_WRITE_THROUGH_NO_ALLOC; // TEX [2:0] = 0, C=1, B=0
+  } else if (Attributes & EFI_MEMORY_WB) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK;
+    // write back (with allocate)
+    EntryValue |= TT_DESCRIPTOR_PAGE_CACHE_POLICY_WRITE_BACK_ALLOC; // TEX [2:0] = 001, C=1, B=1
+  } else if (Attributes & CACHE_ATTRIBUTE_MASK) {
+    // catch unsupported memory type attributes
+    ASSERT (FALSE);
+    return EFI_UNSUPPORTED;
+  }
+
+  if (Attributes & EFI_MEMORY_RO) {
+    EntryValue |= TT_DESCRIPTOR_PAGE_AP_RO_RO;
+  } else {
+    EntryValue |= TT_DESCRIPTOR_PAGE_AP_RW_RW;
+  }
+
+  // Obtain page table base
+  FirstLevelTable = (ARM_FIRST_LEVEL_DESCRIPTOR *)ArmGetTTBR0BaseAddress ();
+
+  // Calculate number of 4KB page table entries to change
+  NumPageEntries = Length / TT_DESCRIPTOR_PAGE_SIZE;
+
+  // Iterate for the number of 4KB pages to change
+  Offset = 0;
+  for(p = 0; p < NumPageEntries; p++) {
+    // Calculate index into first level translation table for page table value
+
+    FirstLevelIdx = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(BaseAddress + Offset) >> TT_DESCRIPTOR_SECTION_BASE_SHIFT;
+    ASSERT (FirstLevelIdx < TRANSLATION_TABLE_SECTION_COUNT);
+
+    // Read the descriptor from the first level page table
+    Descriptor = FirstLevelTable[FirstLevelIdx];
+
+    // Does this descriptor need to be converted from section entry to 4K pages?
+    if (!TT_DESCRIPTOR_SECTION_TYPE_IS_PAGE_TABLE(Descriptor)) {
+      Status = ConvertSectionToPages (FirstLevelIdx << TT_DESCRIPTOR_SECTION_BASE_SHIFT);
+      if (EFI_ERROR(Status)) {
+        // Exit for loop
+        break;
+      }
+
+      // Re-read descriptor
+      Descriptor = FirstLevelTable[FirstLevelIdx];
+      if (FlushTlbs != NULL) {
+        *FlushTlbs = TRUE;
+      }
+    }
+
+    // Obtain page table base address
+    PageTable = (ARM_PAGE_TABLE_ENTRY *)TT_DESCRIPTOR_PAGE_BASE_ADDRESS(Descriptor);
+
+    // Calculate index into the page table
+    PageTableIndex = ((BaseAddress + Offset) & TT_DESCRIPTOR_PAGE_INDEX_MASK) >> TT_DESCRIPTOR_PAGE_BASE_SHIFT;
+    ASSERT (PageTableIndex < TRANSLATION_TABLE_PAGE_COUNT);
+
+    // Get the entry
+    CurrentPageTableEntry = PageTable[PageTableIndex];
+
+    // Mask off appropriate fields
+    PageTableEntry = CurrentPageTableEntry & ~EntryMask;
+
+    // Mask in new attributes and/or permissions
+    PageTableEntry |= EntryValue;
+
+    if (CurrentPageTableEntry  != PageTableEntry) {
+      Mva = (VOID *)(UINTN)((((UINTN)FirstLevelIdx) << TT_DESCRIPTOR_SECTION_BASE_SHIFT) + (PageTableIndex << TT_DESCRIPTOR_PAGE_BASE_SHIFT));
+
+      // Only need to update if we are changing the entry
+      PageTable[PageTableIndex] = PageTableEntry;
+      ArmUpdateTranslationTableEntry ((VOID *)&PageTable[PageTableIndex], Mva);
+    }
+
+    Status = EFI_SUCCESS;
+    Offset += TT_DESCRIPTOR_PAGE_SIZE;
+
+  } // End first level translation table loop
+
+  return Status;
+}
+
+STATIC
+EFI_STATUS
+UpdateSectionEntries (
+  IN EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN UINT64                    Length,
+  IN UINT64                    Attributes
+  )
+{
+  EFI_STATUS    Status = EFI_SUCCESS;
+  UINT32        EntryMask;
+  UINT32        EntryValue;
+  UINT32        FirstLevelIdx;
+  UINT32        NumSections;
+  UINT32        i;
+  UINT32        CurrentDescriptor;
+  UINT32        Descriptor;
+  VOID          *Mva;
+  volatile ARM_FIRST_LEVEL_DESCRIPTOR   *FirstLevelTable;
+
+  // EntryMask: bitmask of values to change (1 = change this value, 0 = leave alone)
+  // EntryValue: values at bit positions specified by EntryMask
+
+  // Make sure we handle a section range that is unmapped
+  EntryMask = TT_DESCRIPTOR_SECTION_TYPE_MASK | TT_DESCRIPTOR_SECTION_XN_MASK |
+              TT_DESCRIPTOR_SECTION_AP_MASK;
+  EntryValue = TT_DESCRIPTOR_SECTION_TYPE_SECTION;
+
+  // Although the PI spec is unclear on this, the GCD guarantees that only
+  // one Attribute bit is set at a time, so the order of the conditionals below
+  // is irrelevant. If no memory attribute is specified, we preserve whatever
+  // memory type is set in the page tables, and update the permission attributes
+  // only.
+  if (Attributes & EFI_MEMORY_UC) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
+    // map to strongly ordered
+    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_STRONGLY_ORDERED; // TEX[2:0] = 0, C=0, B=0
+  } else if (Attributes & EFI_MEMORY_WC) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
+    // map to normal non-cachable
+    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_NON_CACHEABLE; // TEX [2:0]= 001 = 0x2, B=0, C=0
+  } else if (Attributes & EFI_MEMORY_WT) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
+    // write through with no-allocate
+    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_WRITE_THROUGH_NO_ALLOC; // TEX [2:0] = 0, C=1, B=0
+  } else if (Attributes & EFI_MEMORY_WB) {
+    // modify cacheability attributes
+    EntryMask |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK;
+    // write back (with allocate)
+    EntryValue |= TT_DESCRIPTOR_SECTION_CACHE_POLICY_WRITE_BACK_ALLOC; // TEX [2:0] = 001, C=1, B=1
+  } else if (Attributes & CACHE_ATTRIBUTE_MASK) {
+    // catch unsupported memory type attributes
+    ASSERT (FALSE);
+    return EFI_UNSUPPORTED;
+  }
+
+  if (Attributes & EFI_MEMORY_RO) {
+    EntryValue |= TT_DESCRIPTOR_SECTION_AP_RO_RO;
+  } else {
+    EntryValue |= TT_DESCRIPTOR_SECTION_AP_RW_RW;
+  }
+
+  if (Attributes & EFI_MEMORY_XP) {
+    EntryValue |= TT_DESCRIPTOR_SECTION_XN_MASK;
+  }
+
+  // obtain page table base
+  FirstLevelTable = (ARM_FIRST_LEVEL_DESCRIPTOR *)ArmGetTTBR0BaseAddress ();
+
+  // calculate index into first level translation table for start of modification
+  FirstLevelIdx = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(BaseAddress) >> TT_DESCRIPTOR_SECTION_BASE_SHIFT;
+  ASSERT (FirstLevelIdx < TRANSLATION_TABLE_SECTION_COUNT);
+
+  // calculate number of 1MB first level entries this applies to
+  NumSections = Length / TT_DESCRIPTOR_SECTION_SIZE;
+
+  // iterate through each descriptor
+  for(i=0; i<NumSections; i++) {
+    CurrentDescriptor = FirstLevelTable[FirstLevelIdx + i];
+
+    // has this descriptor already been converted to pages?
+    if (TT_DESCRIPTOR_SECTION_TYPE_IS_PAGE_TABLE(CurrentDescriptor)) {
+      // forward this 1MB range to page table function instead
+      Status = UpdatePageEntries (
+                 (FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT,
+                 TT_DESCRIPTOR_SECTION_SIZE,
+                 Attributes,
+                 NULL);
+    } else {
+      // still a section entry
+
+      if (CurrentDescriptor != 0) {
+        // mask off appropriate fields
+        Descriptor = CurrentDescriptor & ~EntryMask;
+      } else {
+        Descriptor = ((UINTN)FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT;
+      }
+
+      // mask in new attributes and/or permissions
+      Descriptor |= EntryValue;
+
+      if (CurrentDescriptor  != Descriptor) {
+        Mva = (VOID *)(UINTN)(((UINTN)FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT);
+
+        // Only need to update if we are changing the descriptor
+        FirstLevelTable[FirstLevelIdx + i] = Descriptor;
+        ArmUpdateTranslationTableEntry ((VOID *)&FirstLevelTable[FirstLevelIdx + i], Mva);
+      }
+
+      Status = EFI_SUCCESS;
+    }
+  }
+
+  return Status;
+}
+
+EFI_STATUS
+ArmSetMemoryAttributes (
+  IN EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN UINT64                    Length,
+  IN UINT64                    Attributes
+  )
+{
+  EFI_STATUS    Status;
+  UINT64        ChunkLength;
+  BOOLEAN       FlushTlbs;
+
+  if (BaseAddress > (UINT64)MAX_ADDRESS) {
+    return EFI_UNSUPPORTED;
+  }
+
+  Length = MIN (Length, (UINT64)MAX_ADDRESS - BaseAddress + 1);
+  if (Length == 0) {
+    return EFI_SUCCESS;
+  }
+
+  FlushTlbs = FALSE;
+  while (Length > 0) {
+    if ((BaseAddress % TT_DESCRIPTOR_SECTION_SIZE == 0) &&
+        Length >= TT_DESCRIPTOR_SECTION_SIZE) {
+
+      ChunkLength = Length - Length % TT_DESCRIPTOR_SECTION_SIZE;
+
+      DEBUG ((DEBUG_PAGE,
+        "SetMemoryAttributes(): MMU section 0x%lx length 0x%lx to %lx\n",
+        BaseAddress, ChunkLength, Attributes));
+
+      Status = UpdateSectionEntries (BaseAddress, ChunkLength, Attributes);
+
+      FlushTlbs = TRUE;
+    } else {
+
+      //
+      // Process page by page until the next section boundary, but only if
+      // we have more than a section's worth of area to deal with after that.
+      //
+      ChunkLength = TT_DESCRIPTOR_SECTION_SIZE -
+                    (BaseAddress % TT_DESCRIPTOR_SECTION_SIZE);
+      if (ChunkLength + TT_DESCRIPTOR_SECTION_SIZE > Length) {
+        ChunkLength = Length;
+      }
+
+      DEBUG ((DEBUG_PAGE,
+        "SetMemoryAttributes(): MMU page 0x%lx length 0x%lx to %lx\n",
+        BaseAddress, ChunkLength, Attributes));
+
+      Status = UpdatePageEntries (BaseAddress, ChunkLength, Attributes,
+                 &FlushTlbs);
+    }
+
+    if (EFI_ERROR (Status)) {
+      break;
+    }
+
+    BaseAddress += ChunkLength;
+    Length -= ChunkLength;
+  }
+
+  if (FlushTlbs) {
+    ArmInvalidateTlb ();
+  }
+  return Status;
+}
+
+EFI_STATUS
+ArmSetMemoryRegionNoExec (
+  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN  UINT64                    Length
+  )
+{
+  return ArmSetMemoryAttributes (BaseAddress, Length, EFI_MEMORY_XP);
+}
+
+EFI_STATUS
+ArmClearMemoryRegionNoExec (
+  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN  UINT64                    Length
+  )
+{
+  return ArmSetMemoryAttributes (BaseAddress, Length, __EFI_MEMORY_RWX);
+}
+
+EFI_STATUS
+ArmSetMemoryRegionReadOnly (
+  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN  UINT64                    Length
+  )
+{
+  return ArmSetMemoryAttributes (BaseAddress, Length, EFI_MEMORY_RO);
+}
+
+EFI_STATUS
+ArmClearMemoryRegionReadOnly (
+  IN  EFI_PHYSICAL_ADDRESS      BaseAddress,
+  IN  UINT64                    Length
+  )
+{
+  return ArmSetMemoryAttributes (BaseAddress, Length, __EFI_MEMORY_RWX);
+}

ArmPkg/Library/ArmMmuLib/ArmMmuBaseLib.inf

@@ -14,6 +14,8 @@
   MODULE_TYPE                    = BASE
   VERSION_STRING                 = 1.0
   LIBRARY_CLASS                  = ArmMmuLib
+
+[Defines.AARCH64]
   CONSTRUCTOR                    = ArmMmuBaseLibConstructor
 
 [Sources.AARCH64]
@@ -21,7 +23,9 @@
   AArch64/ArmMmuLibReplaceEntry.S
 
 [Sources.ARM]
+  Arm/ArmMmuLibConvert.c
   Arm/ArmMmuLibCore.c
+  Arm/ArmMmuLibUpdate.c
   Arm/ArmMmuLibV7Support.S   |GCC
   Arm/ArmMmuLibV7Support.asm |RVCT
 

ArmPkg/Library/ArmSmcLibNull/ArmSmcLibNull.c

@@ -1,16 +1,16 @@
-//
-//  Copyright (c) 2016, Linaro Limited. All rights reserved.
-//
-//  SPDX-License-Identifier: BSD-2-Clause-Patent
-//
-//
-
-#include <Base.h>
-#include <Library/ArmSmcLib.h>
-
-VOID
-ArmCallSmc (
-  IN OUT ARM_SMC_ARGS *Args
-  )
-{
-}
+//
+//  Copyright (c) 2016, Linaro Limited. All rights reserved.
+//
+//  SPDX-License-Identifier: BSD-2-Clause-Patent
+//
+//
+
+#include <Base.h>
+#include <Library/ArmSmcLib.h>
+
+VOID
+ArmCallSmc (
+  IN OUT ARM_SMC_ARGS *Args
+  )
+{
+}

ArmPkg/Library/ArmSoftFloatLib/ArmSoftFloatLib.c

@@ -1,283 +1,283 @@
-/*
- * Copyright (c) 2015 - 2019, Linaro Limited
- *
- * SPDX-License-Identifier: BSD-2-Clause-Patent
- */
-
-#include "platform.h"
-#include <softfloat.h>
-
-/*
- * On ARM32 EABI defines both a soft-float ABI and a hard-float ABI,
- * hard-float is basically a super set of soft-float. Hard-float requires
- * all the support routines provided for soft-float, but the compiler may
- * choose to optimize to not use some of them.
- *
- * The AEABI functions uses soft-float calling convention even if the
- * functions are compiled for hard-float. So where float and double would
- * have been expected we use aeabi_float_t and aeabi_double_t respectively
- * instead.
- */
-typedef uint32_t aeabi_float_t;
-typedef uint64_t aeabi_double_t;
-
-/*
- * Helpers to convert between float32 and aeabi_float_t, and float64 and
- * aeabi_double_t used by the AEABI functions below.
- */
-static aeabi_float_t f32_to_f(float32_t val)
-{
-	return val.v;
-}
-
-static float32_t f32_from_f(aeabi_float_t val)
-{
-	float32_t res;
-
-	res.v = val;
-
-	return res;
-}
-
-static aeabi_double_t f64_to_d(float64_t val)
-{
-	return val.v;
-}
-
-static float64_t f64_from_d(aeabi_double_t val)
-{
-	float64_t res;
-
-	res.v = val;
-
-	return res;
-}
-
-/*
- * From ARM Run-time ABI for ARM Architecture
- * ARM IHI 0043D, current through ABI release 2.09
- *
- * 4.1.2 The floating-point helper functions
- */
-
-/*
- * Table 2, Standard aeabi_double_t precision floating-point arithmetic helper
- * functions
- */
-
-aeabi_double_t __aeabi_dadd(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_to_d(f64_add(f64_from_d(a), f64_from_d(b)));
-}
-
-aeabi_double_t __aeabi_ddiv(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_to_d(f64_div(f64_from_d(a), f64_from_d(b)));
-}
-
-aeabi_double_t __aeabi_dmul(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_to_d(f64_mul(f64_from_d(a), f64_from_d(b)));
-}
-
-
-aeabi_double_t __aeabi_drsub(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_to_d(f64_sub(f64_from_d(b), f64_from_d(a)));
-}
-
-aeabi_double_t __aeabi_dsub(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_to_d(f64_sub(f64_from_d(a), f64_from_d(b)));
-}
-
-/*
- * Table 3, double precision floating-point comparison helper functions
- */
-
-int __aeabi_dcmpeq(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_eq(f64_from_d(a), f64_from_d(b));
-}
-
-int __aeabi_dcmplt(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_lt(f64_from_d(a), f64_from_d(b));
-}
-
-int __aeabi_dcmple(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_le(f64_from_d(a), f64_from_d(b));
-}
-
-int __aeabi_dcmpge(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_le(f64_from_d(b), f64_from_d(a));
-}
-
-int __aeabi_dcmpgt(aeabi_double_t a, aeabi_double_t b)
-{
-	return f64_lt(f64_from_d(b), f64_from_d(a));
-}
-
-/*
- * Table 4, Standard single precision floating-point arithmetic helper
- * functions
- */
-
-aeabi_float_t __aeabi_fadd(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_to_f(f32_add(f32_from_f(a), f32_from_f(b)));
-}
-
-aeabi_float_t __aeabi_fdiv(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_to_f(f32_div(f32_from_f(a), f32_from_f(b)));
-}
-
-aeabi_float_t __aeabi_fmul(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_to_f(f32_mul(f32_from_f(a), f32_from_f(b)));
-}
-
-aeabi_float_t __aeabi_frsub(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_to_f(f32_sub(f32_from_f(b), f32_from_f(a)));
-}
-
-aeabi_float_t __aeabi_fsub(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_to_f(f32_sub(f32_from_f(a), f32_from_f(b)));
-}
-
-/*
- * Table 5, Standard single precision floating-point comparison helper
- * functions
- */
-
-int __aeabi_fcmpeq(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_eq(f32_from_f(a), f32_from_f(b));
-}
-
-int __aeabi_fcmplt(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_lt(f32_from_f(a), f32_from_f(b));
-}
-
-int __aeabi_fcmple(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_le(f32_from_f(a), f32_from_f(b));
-}
-
-int __aeabi_fcmpge(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_le(f32_from_f(b), f32_from_f(a));
-}
-
-int __aeabi_fcmpgt(aeabi_float_t a, aeabi_float_t b)
-{
-	return f32_lt(f32_from_f(b), f32_from_f(a));
-}
-
-/*
- * Table 6, Standard floating-point to integer conversions
- */
-
-int __aeabi_d2iz(aeabi_double_t a)
-{
-	return f64_to_i32_r_minMag(f64_from_d(a), false);
-}
-
-unsigned __aeabi_d2uiz(aeabi_double_t a)
-{
-	return f64_to_ui32_r_minMag(f64_from_d(a), false);
-}
-
-long long __aeabi_d2lz(aeabi_double_t a)
-{
-	return f64_to_i64_r_minMag(f64_from_d(a), false);
-}
-
-unsigned long long __aeabi_d2ulz(aeabi_double_t a)
-{
-	return f64_to_ui64_r_minMag(f64_from_d(a), false);
-}
-
-int __aeabi_f2iz(aeabi_float_t a)
-{
-	return f32_to_i32_r_minMag(f32_from_f(a), false);
-}
-
-unsigned __aeabi_f2uiz(aeabi_float_t a)
-{
-	return f32_to_ui32_r_minMag(f32_from_f(a), false);
-}
-
-long long __aeabi_f2lz(aeabi_float_t a)
-{
-	return f32_to_i64_r_minMag(f32_from_f(a), false);
-}
-
-unsigned long long __aeabi_f2ulz(aeabi_float_t a)
-{
-	return f32_to_ui64_r_minMag(f32_from_f(a), false);
-}
-
-/*
- * Table 7, Standard conversions between floating types
- */
-
-aeabi_float_t __aeabi_d2f(aeabi_double_t a)
-{
-	return f32_to_f(f64_to_f32(f64_from_d(a)));
-}
-
-aeabi_double_t __aeabi_f2d(aeabi_float_t a)
-{
-	return f64_to_d(f32_to_f64(f32_from_f(a)));
-}
-
-/*
- * Table 8, Standard integer to floating-point conversions
- */
-
-aeabi_double_t __aeabi_i2d(int a)
-{
-	return f64_to_d(i32_to_f64(a));
-}
-
-aeabi_double_t __aeabi_ui2d(unsigned a)
-{
-	return f64_to_d(ui32_to_f64(a));
-}
-
-aeabi_double_t __aeabi_l2d(long long a)
-{
-	return f64_to_d(i64_to_f64(a));
-}
-
-aeabi_double_t __aeabi_ul2d(unsigned long long a)
-{
-	return f64_to_d(ui64_to_f64(a));
-}
-
-aeabi_float_t __aeabi_i2f(int a)
-{
-	return f32_to_f(i32_to_f32(a));
-}
-
-aeabi_float_t __aeabi_ui2f(unsigned a)
-{
-	return f32_to_f(ui32_to_f32(a));
-}
-
-aeabi_float_t __aeabi_l2f(long long a)
-{
-	return f32_to_f(i64_to_f32(a));
-}
-
-aeabi_float_t __aeabi_ul2f(unsigned long long a)
-{
-	return f32_to_f(ui64_to_f32(a));
-}
+/*
+ * Copyright (c) 2015 - 2019, Linaro Limited
+ *
+ * SPDX-License-Identifier: BSD-2-Clause-Patent
+ */
+
+#include "platform.h"
+#include <softfloat.h>
+
+/*
+ * On ARM32 EABI defines both a soft-float ABI and a hard-float ABI,
+ * hard-float is basically a super set of soft-float. Hard-float requires
+ * all the support routines provided for soft-float, but the compiler may
+ * choose to optimize to not use some of them.
+ *
+ * The AEABI functions uses soft-float calling convention even if the
+ * functions are compiled for hard-float. So where float and double would
+ * have been expected we use aeabi_float_t and aeabi_double_t respectively
+ * instead.
+ */
+typedef uint32_t aeabi_float_t;
+typedef uint64_t aeabi_double_t;
+
+/*
+ * Helpers to convert between float32 and aeabi_float_t, and float64 and
+ * aeabi_double_t used by the AEABI functions below.
+ */
+static aeabi_float_t f32_to_f(float32_t val)
+{
+  return val.v;
+}
+
+static float32_t f32_from_f(aeabi_float_t val)
+{
+  float32_t res;
+
+  res.v = val;
+
+  return res;
+}
+
+static aeabi_double_t f64_to_d(float64_t val)
+{
+  return val.v;
+}
+
+static float64_t f64_from_d(aeabi_double_t val)
+{
+  float64_t res;
+
+  res.v = val;
+
+  return res;
+}
+
+/*
+ * From ARM Run-time ABI for ARM Architecture
+ * ARM IHI 0043D, current through ABI release 2.09
+ *
+ * 4.1.2 The floating-point helper functions
+ */
+
+/*
+ * Table 2, Standard aeabi_double_t precision floating-point arithmetic helper
+ * functions
+ */
+
+aeabi_double_t __aeabi_dadd(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_to_d(f64_add(f64_from_d(a), f64_from_d(b)));
+}
+
+aeabi_double_t __aeabi_ddiv(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_to_d(f64_div(f64_from_d(a), f64_from_d(b)));
+}
+
+aeabi_double_t __aeabi_dmul(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_to_d(f64_mul(f64_from_d(a), f64_from_d(b)));
+}
+
+
+aeabi_double_t __aeabi_drsub(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_to_d(f64_sub(f64_from_d(b), f64_from_d(a)));
+}
+
+aeabi_double_t __aeabi_dsub(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_to_d(f64_sub(f64_from_d(a), f64_from_d(b)));
+}
+
+/*
+ * Table 3, double precision floating-point comparison helper functions
+ */
+
+int __aeabi_dcmpeq(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_eq(f64_from_d(a), f64_from_d(b));
+}
+
+int __aeabi_dcmplt(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_lt(f64_from_d(a), f64_from_d(b));
+}
+
+int __aeabi_dcmple(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_le(f64_from_d(a), f64_from_d(b));
+}
+
+int __aeabi_dcmpge(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_le(f64_from_d(b), f64_from_d(a));
+}
+
+int __aeabi_dcmpgt(aeabi_double_t a, aeabi_double_t b)
+{
+  return f64_lt(f64_from_d(b), f64_from_d(a));
+}
+
+/*
+ * Table 4, Standard single precision floating-point arithmetic helper
+ * functions
+ */
+
+aeabi_float_t __aeabi_fadd(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_to_f(f32_add(f32_from_f(a), f32_from_f(b)));
+}
+
+aeabi_float_t __aeabi_fdiv(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_to_f(f32_div(f32_from_f(a), f32_from_f(b)));
+}
+
+aeabi_float_t __aeabi_fmul(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_to_f(f32_mul(f32_from_f(a), f32_from_f(b)));
+}
+
+aeabi_float_t __aeabi_frsub(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_to_f(f32_sub(f32_from_f(b), f32_from_f(a)));
+}
+
+aeabi_float_t __aeabi_fsub(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_to_f(f32_sub(f32_from_f(a), f32_from_f(b)));
+}
+
+/*
+ * Table 5, Standard single precision floating-point comparison helper
+ * functions
+ */
+
+int __aeabi_fcmpeq(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_eq(f32_from_f(a), f32_from_f(b));
+}
+
+int __aeabi_fcmplt(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_lt(f32_from_f(a), f32_from_f(b));
+}
+
+int __aeabi_fcmple(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_le(f32_from_f(a), f32_from_f(b));
+}
+
+int __aeabi_fcmpge(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_le(f32_from_f(b), f32_from_f(a));
+}
+
+int __aeabi_fcmpgt(aeabi_float_t a, aeabi_float_t b)
+{
+  return f32_lt(f32_from_f(b), f32_from_f(a));
+}
+
+/*
+ * Table 6, Standard floating-point to integer conversions
+ */
+
+int __aeabi_d2iz(aeabi_double_t a)
+{
+  return f64_to_i32_r_minMag(f64_from_d(a), false);
+}
+
+unsigned __aeabi_d2uiz(aeabi_double_t a)
+{
+  return f64_to_ui32_r_minMag(f64_from_d(a), false);
+}
+
+long long __aeabi_d2lz(aeabi_double_t a)
+{
+  return f64_to_i64_r_minMag(f64_from_d(a), false);
+}
+
+unsigned long long __aeabi_d2ulz(aeabi_double_t a)
+{
+  return f64_to_ui64_r_minMag(f64_from_d(a), false);
+}
+
+int __aeabi_f2iz(aeabi_float_t a)
+{
+  return f32_to_i32_r_minMag(f32_from_f(a), false);
+}
+
+unsigned __aeabi_f2uiz(aeabi_float_t a)
+{
+  return f32_to_ui32_r_minMag(f32_from_f(a), false);
+}
+
+long long __aeabi_f2lz(aeabi_float_t a)
+{
+  return f32_to_i64_r_minMag(f32_from_f(a), false);
+}
+
+unsigned long long __aeabi_f2ulz(aeabi_float_t a)
+{
+  return f32_to_ui64_r_minMag(f32_from_f(a), false);
+}
+
+/*
+ * Table 7, Standard conversions between floating types
+ */
+
+aeabi_float_t __aeabi_d2f(aeabi_double_t a)
+{
+  return f32_to_f(f64_to_f32(f64_from_d(a)));
+}
+
+aeabi_double_t __aeabi_f2d(aeabi_float_t a)
+{
+  return f64_to_d(f32_to_f64(f32_from_f(a)));
+}
+
+/*
+ * Table 8, Standard integer to floating-point conversions
+ */
+
+aeabi_double_t __aeabi_i2d(int a)
+{
+  return f64_to_d(i32_to_f64(a));
+}
+
+aeabi_double_t __aeabi_ui2d(unsigned a)
+{
+  return f64_to_d(ui32_to_f64(a));
+}
+
+aeabi_double_t __aeabi_l2d(long long a)
+{
+  return f64_to_d(i64_to_f64(a));
+}
+
+aeabi_double_t __aeabi_ul2d(unsigned long long a)
+{
+  return f64_to_d(ui64_to_f64(a));
+}
+
+aeabi_float_t __aeabi_i2f(int a)
+{
+  return f32_to_f(i32_to_f32(a));
+}
+
+aeabi_float_t __aeabi_ui2f(unsigned a)
+{
+  return f32_to_f(ui32_to_f32(a));
+}
+
+aeabi_float_t __aeabi_l2f(long long a)
+{
+  return f32_to_f(i64_to_f32(a));
+}
+
+aeabi_float_t __aeabi_ul2f(unsigned long long a)
+{
+  return f32_to_f(ui64_to_f32(a));
+}

ArmPkg/Library/CompilerIntrinsicsLib/AArch64/Atomics.S

@@ -0,0 +1,142 @@
+#------------------------------------------------------------------------------
+#
+# Copyright (c) 2020, Arm, Limited. All rights reserved.<BR>
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+#------------------------------------------------------------------------------
+
+        /*
+         * Provide the GCC intrinsics that are required when using GCC 9 or
+         * later with the -moutline-atomics options (which became the default
+         * in GCC 10)
+         */
+        .arch armv8-a
+
+        .macro          reg_alias, pfx, sz
+        r0_\sz          .req    \pfx\()0
+        r1_\sz          .req    \pfx\()1
+        tmp0_\sz        .req    \pfx\()16
+        tmp1_\sz        .req    \pfx\()17
+        .endm
+
+        /*
+         * Define register aliases of the right type for each size
+         * (xN for 8 bytes, wN for everything smaller)
+         */
+        reg_alias       w, 1
+        reg_alias       w, 2
+        reg_alias       w, 4
+        reg_alias       x, 8
+
+        .macro          fn_start, name:req
+        .section        .text.\name
+        .globl          \name
+        .type           \name, %function
+\name\():
+        .endm
+
+        .macro          fn_end, name:req
+        .size           \name, . - \name
+        .endm
+
+        /*
+         * Emit an atomic helper for \model with operands of size \sz, using
+         * the operation specified by \insn (which is the LSE name), and which
+         * can be implemented using the generic load-locked/store-conditional
+         * (LL/SC) sequence below, using the arithmetic operation given by
+         * \opc.
+         */
+        .macro          emit_ld_sz, sz:req, insn:req, opc:req, model:req, s, a, l
+        fn_start        __aarch64_\insn\()\sz\()\model
+        mov             tmp0_\sz, r0_\sz
+0:      ld\a\()xr\s     r0_\sz, [x1]
+        .ifnc           \insn, swp
+        \opc            tmp1_\sz, r0_\sz, tmp0_\sz
+        st\l\()xr\s     w15, tmp1_\sz, [x1]
+        .else
+        st\l\()xr\s     w15, tmp0_\sz, [x1]
+        .endif
+        cbnz            w15, 0b
+        ret
+        fn_end          __aarch64_\insn\()\sz\()\model
+        .endm
+
+        /*
+         * Emit atomic helpers for \model for operand sizes in the
+         * set {1, 2, 4, 8}, for the instruction pattern given by
+         * \insn. (This is the LSE name, but this implementation uses
+         * the generic LL/SC sequence using \opc as the arithmetic
+         * operation on the target.)
+         */
+        .macro          emit_ld, insn:req, opc:req, model:req, a, l
+        emit_ld_sz      1, \insn, \opc, \model, b, \a, \l
+        emit_ld_sz      2, \insn, \opc, \model, h, \a, \l
+        emit_ld_sz      4, \insn, \opc, \model,  , \a, \l
+        emit_ld_sz      8, \insn, \opc, \model,  , \a, \l
+        .endm
+
+        /*
+         * Emit the compare and swap helper for \model and size \sz
+         * using LL/SC instructions.
+         */
+        .macro          emit_cas_sz, sz:req, model:req, uxt:req, s, a, l
+        fn_start        __aarch64_cas\sz\()\model
+        \uxt            tmp0_\sz, r0_\sz
+0:      ld\a\()xr\s     r0_\sz, [x2]
+        cmp             r0_\sz, tmp0_\sz
+        bne             1f
+        st\l\()xr\s     w15, r1_\sz, [x2]
+        cbnz            w15, 0b
+1:      ret
+        fn_end          __aarch64_cas\sz\()\model
+        .endm
+
+        /*
+         * Emit compare-and-swap helpers for \model for operand sizes in the
+         * set {1, 2, 4, 8, 16}.
+         */
+        .macro          emit_cas, model:req, a, l
+        emit_cas_sz     1, \model, uxtb, b, \a, \l
+        emit_cas_sz     2, \model, uxth, h, \a, \l
+        emit_cas_sz     4, \model, mov ,  , \a, \l
+        emit_cas_sz     8, \model, mov ,  , \a, \l
+
+        /*
+         * We cannot use the parameterized sequence for 16 byte CAS, so we
+         * need to define it explicitly.
+         */
+        fn_start        __aarch64_cas16\model
+        mov             x16, x0
+        mov             x17, x1
+0:      ld\a\()xp       x0, x1, [x4]
+        cmp             x0, x16
+        ccmp            x1, x17, #0, eq
+        bne             1f
+        st\l\()xp       w15, x16, x17, [x4]
+        cbnz            w15, 0b
+1:      ret
+        fn_end          __aarch64_cas16\model
+        .endm
+
+        /*
+         * Emit the set of GCC outline atomic helper functions for
+         * the memory ordering model given by \model:
+         * - relax      unordered loads and stores
+         * - acq        load-acquire, unordered store
+         * - rel        unordered load, store-release
+         * - acq_rel    load-acquire, store-release
+         */
+        .macro          emit_model, model:req, a, l
+        emit_ld         ldadd, add, \model, \a, \l
+        emit_ld         ldclr, bic, \model, \a, \l
+        emit_ld         ldeor, eor, \model, \a, \l
+        emit_ld         ldset, orr, \model, \a, \l
+        emit_ld         swp,   mov, \model, \a, \l
+        emit_cas        \model, \a, \l
+        .endm
+
+        emit_model      _relax
+        emit_model      _acq, a
+        emit_model      _rel,, l
+        emit_model      _acq_rel, a, l

ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf

@@ -79,6 +79,9 @@
   Arm/ldivmod.asm      | MSFT
   Arm/llsr.asm         | MSFT
 
+[Sources.AARCH64]
+  AArch64/Atomics.S    | GCC
+
 [Packages]
   MdePkg/MdePkg.dec
   ArmPkg/ArmPkg.dec

ArmPkg/Library/CompilerIntrinsicsLib/memset.c

@@ -1,55 +1,55 @@
-//------------------------------------------------------------------------------
-//
-// Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
-//
-// SPDX-License-Identifier: BSD-2-Clause-Patent
-//
-//------------------------------------------------------------------------------
-
-typedef __SIZE_TYPE__ size_t;
-
-static __attribute__((__used__))
-void *__memset(void *s, int c, size_t n)
-{
-  unsigned char *d = s;
-
-  while (n--)
-    *d++ = c;
-
-  return s;
-}
-
-//
-// Other modules (such as CryptoPkg/IntrinsicLib) may provide another
-// implementation of memset(), which may conflict with this one if this
-// object was pulled into the link due to the definitions below. So make
-// our memset() 'weak' to let the other implementation take precedence.
-//
-__attribute__((__weak__, __alias__("__memset")))
-void *memset(void *dest, int c, size_t n);
-
-#ifdef __arm__
-
-void __aeabi_memset(void *dest, size_t n, int c)
-{
-  __memset(dest, c, n);
-}
-
-__attribute__((__alias__("__aeabi_memset")))
-void __aeabi_memset4(void *dest, size_t n, int c);
-
-__attribute__((__alias__("__aeabi_memset")))
-void __aeabi_memset8(void *dest, size_t n, int c);
-
-void __aeabi_memclr(void *dest, size_t n)
-{
-  __memset(dest, 0, n);
-}
-
-__attribute__((__alias__("__aeabi_memclr")))
-void __aeabi_memclr4(void *dest, size_t n);
-
-__attribute__((__alias__("__aeabi_memclr")))
-void __aeabi_memclr8(void *dest, size_t n);
-
-#endif
+//------------------------------------------------------------------------------
+//
+// Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
+//
+// SPDX-License-Identifier: BSD-2-Clause-Patent
+//
+//------------------------------------------------------------------------------
+
+typedef __SIZE_TYPE__ size_t;
+
+static __attribute__((__used__))
+void *__memset(void *s, int c, size_t n)
+{
+  unsigned char *d = s;
+
+  while (n--)
+    *d++ = c;
+
+  return s;
+}
+
+//
+// Other modules (such as CryptoPkg/IntrinsicLib) may provide another
+// implementation of memset(), which may conflict with this one if this
+// object was pulled into the link due to the definitions below. So make
+// our memset() 'weak' to let the other implementation take precedence.
+//
+__attribute__((__weak__, __alias__("__memset")))
+void *memset(void *dest, int c, size_t n);
+
+#ifdef __arm__
+
+void __aeabi_memset(void *dest, size_t n, int c)
+{
+  __memset(dest, c, n);
+}
+
+__attribute__((__alias__("__aeabi_memset")))
+void __aeabi_memset4(void *dest, size_t n, int c);
+
+__attribute__((__alias__("__aeabi_memset")))
+void __aeabi_memset8(void *dest, size_t n, int c);
+
+void __aeabi_memclr(void *dest, size_t n)
+{
+  __memset(dest, 0, n);
+}
+
+__attribute__((__alias__("__aeabi_memclr")))
+void __aeabi_memclr4(void *dest, size_t n);
+
+__attribute__((__alias__("__aeabi_memclr")))
+void __aeabi_memclr8(void *dest, size_t n);
+
+#endif

ArmPkg/Library/GccLto/liblto-aarch64.s

@@ -1,21 +1,21 @@
-//
-// Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
-//
-// SPDX-License-Identifier: BSD-2-Clause-Patent
-//
-
-//
-// GCC in LTO mode interoperates poorly with non-standard libraries that
-// provide implementations of compiler intrinsics such as memcpy/memset
-// or the stack protector entry points.
-//
-// By referencing these functions from a non-LTO object that can be passed
-// to the linker via the -plugin-opt=-pass-through=-lxxx options, the
-// intrinsics are included in the link in a way that allows them to be
-// pruned again if no other references to them exist.
-//
-
-	.long	memcpy - .
-	.long	memset - .
-	.long	__stack_chk_fail - .
-	.long	__stack_chk_guard - .
+//
+// Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
+//
+// SPDX-License-Identifier: BSD-2-Clause-Patent
+//
+
+//
+// GCC in LTO mode interoperates poorly with non-standard libraries that
+// provide implementations of compiler intrinsics such as memcpy/memset
+// or the stack protector entry points.
+//
+// By referencing these functions from a non-LTO object that can be passed
+// to the linker via the -plugin-opt=-pass-through=-lxxx options, the
+// intrinsics are included in the link in a way that allows them to be
+// pruned again if no other references to them exist.
+//
+
+  .long memcpy - .
+  .long memset - .
+  .long __stack_chk_fail - .
+  .long __stack_chk_guard - .

ArmPkg/Library/GccLto/liblto-arm.s

@@ -1,55 +1,55 @@
-//
-// Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
-//
-// SPDX-License-Identifier: BSD-2-Clause-Patent
-//
-
-//
-// GCC in LTO mode interoperates poorly with non-standard libraries that
-// provide implementations of compiler intrinsics such as memcpy/memset
-// or the stack protector entry points.
-//
-// By referencing these functions from a non-LTO object that can be passed
-// to the linker via the -plugin-opt=-pass-through=-lxxx options, the
-// intrinsics are included in the link in a way that allows them to be
-// pruned again if no other references to them exist.
-//
-
-	.long	memcpy - .
-	.long	memset - .
-	.long	__stack_chk_fail - .
-	.long	__stack_chk_guard - .
-	.long __ashrdi3 - .
-	.long __ashldi3 - .
-	.long __aeabi_idiv - .
-	.long __aeabi_idivmod - .
-	.long __aeabi_uidiv - .
-	.long __aeabi_uidivmod - .
-	.long __divdi3 - .
-	.long __divsi3 - .
-	.long __lshrdi3 - .
-	.long __aeabi_memcpy - .
-	.long __aeabi_memset - .
-	.long memmove - .
-	.long __modsi3 - .
-	.long __moddi3 - .
-	.long __muldi3 - .
-	.long __aeabi_lmul - .
-	.long __ARM_ll_mullu - .
-	.long __udivsi3 - .
-	.long __umodsi3 - .
-	.long __udivdi3 - .
-	.long __umoddi3 - .
-	.long __udivmoddi4 - .
-	.long __clzsi2 - .
-	.long __ctzsi2 - .
-	.long __ucmpdi2 - .
-	.long __switch8 - .
-	.long __switchu8 - .
-	.long __switch16 - .
-	.long __switch32 - .
-	.long __aeabi_ulcmp - .
-	.long __aeabi_uldivmod - .
-	.long __aeabi_ldivmod - .
-	.long __aeabi_llsr - .
-	.long __aeabi_llsl - .
+//
+// Copyright (c) 2016, Linaro Ltd. All rights reserved.<BR>
+//
+// SPDX-License-Identifier: BSD-2-Clause-Patent
+//
+
+//
+// GCC in LTO mode interoperates poorly with non-standard libraries that
+// provide implementations of compiler intrinsics such as memcpy/memset
+// or the stack protector entry points.
+//
+// By referencing these functions from a non-LTO object that can be passed
+// to the linker via the -plugin-opt=-pass-through=-lxxx options, the
+// intrinsics are included in the link in a way that allows them to be
+// pruned again if no other references to them exist.
+//
+
+  .long memcpy - .
+  .long memset - .
+  .long __stack_chk_fail - .
+  .long __stack_chk_guard - .
+  .long __ashrdi3 - .
+  .long __ashldi3 - .
+  .long __aeabi_idiv - .
+  .long __aeabi_idivmod - .
+  .long __aeabi_uidiv - .
+  .long __aeabi_uidivmod - .
+  .long __divdi3 - .
+  .long __divsi3 - .
+  .long __lshrdi3 - .
+  .long __aeabi_memcpy - .
+  .long __aeabi_memset - .
+  .long memmove - .
+  .long __modsi3 - .
+  .long __moddi3 - .
+  .long __muldi3 - .
+  .long __aeabi_lmul - .
+  .long __ARM_ll_mullu - .
+  .long __udivsi3 - .
+  .long __umodsi3 - .
+  .long __udivdi3 - .
+  .long __umoddi3 - .
+  .long __udivmoddi4 - .
+  .long __clzsi2 - .
+  .long __ctzsi2 - .
+  .long __ucmpdi2 - .
+  .long __switch8 - .
+  .long __switchu8 - .
+  .long __switch16 - .
+  .long __switch32 - .
+  .long __aeabi_ulcmp - .
+  .long __aeabi_uldivmod - .
+  .long __aeabi_ldivmod - .
+  .long __aeabi_llsr - .
+  .long __aeabi_llsl - .

ArmPlatformPkg/ArmPlatformPkg.dec

@@ -94,10 +94,6 @@
   gArmPlatformTokenSpaceGuid.PcdArmMaliDpBase|0x0|UINT64|0x00000050
   gArmPlatformTokenSpaceGuid.PcdArmMaliDpMemoryRegionLength|0x0|UINT32|0x00000051
 
-  ## PL180 MCI
-  gArmPlatformTokenSpaceGuid.PcdPL180SysMciRegAddress|0x00000000|UINT32|0x00000028
-  gArmPlatformTokenSpaceGuid.PcdPL180MciBaseAddress|0x00000000|UINT32|0x00000029
-
   # Graphics Output Pixel format
   # 0 : PixelRedGreenBlueReserved8BitPerColor
   # 1 : PixelBlueGreenRedReserved8BitPerColor

ArmPlatformPkg/ArmPlatformPkg.dsc

@@ -4,6 +4,7 @@
 # Copyright (c) 2009 - 2010, Apple Inc. All rights reserved.<BR>
 # Copyright (c) 2011 - 2018, ARM Ltd. All rights reserved.<BR>
 # Copyright (c) 2016 - 2017, Linaro Ltd. All rights reserved.<BR>
+# Copyright (c) Microsoft Corporation.<BR>
 #
 #    SPDX-License-Identifier: BSD-2-Clause-Patent
 #
@@ -90,11 +91,11 @@
   ArmPlatformPkg/Drivers/LcdGraphicsOutputDxe/LcdGraphicsOutputDxe.inf
   ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashDxe.inf
   ArmPlatformPkg/Drivers/PL061GpioDxe/PL061GpioDxe.inf
-  ArmPlatformPkg/Drivers/PL180MciDxe/PL180MciDxe.inf
   ArmPlatformPkg/Drivers/SP805WatchdogDxe/SP805WatchdogDxe.inf
 
   ArmPlatformPkg/Library/ArmPlatformLibNull/ArmPlatformLibNull.inf
   ArmPlatformPkg/Library/ArmPlatformStackLib/ArmPlatformStackLib.inf
+  ArmPlatformPkg/Library/HdLcd/HdLcd.inf
   ArmPlatformPkg/Library/LcdHwNullLib/LcdHwNullLib.inf
   ArmPlatformPkg/Library/LcdPlatformNullLib/LcdPlatformNullLib.inf
   ArmPlatformPkg/Library/NorFlashPlatformNullLib/NorFlashPlatformNullLib.inf
@@ -102,6 +103,7 @@
   ArmPlatformPkg/Library/PL011UartClockLib/PL011UartClockLib.inf
   ArmPlatformPkg/Library/PL011UartLib/PL011UartLib.inf
   ArmPlatformPkg/Library/PL031RealTimeClockLib/PL031RealTimeClockLib.inf
+  ArmPlatformPkg/Library/PL111Lcd/PL111Lcd.inf
   ArmPlatformPkg/Library/PrePiHobListPointerLib/PrePiHobListPointerLib.inf
 
   ArmPlatformPkg/MemoryInitPei/MemoryInitPeiLib.inf

ArmPlatformPkg/Drivers/LcdGraphicsOutputDxe/LcdGraphicsOutputDxe.inf

@@ -38,6 +38,7 @@
   UefiLib
 
 [Protocols]
+  gEfiCpuArchProtocolGuid
   gEfiDevicePathProtocolGuid
   gEfiGraphicsOutputProtocolGuid
 

ArmPlatformPkg/Drivers/PL180MciDxe/PL180Mci.c

@@ -1,563 +0,0 @@
-/** @file
-  This file implement the MMC Host Protocol for the ARM PrimeCell PL180.
-
-  Copyright (c) 2011-2012, ARM Limited. All rights reserved.
-
-  SPDX-License-Identifier: BSD-2-Clause-Patent
-
-**/
-
-#include "PL180Mci.h"
-
-#include <Library/DevicePathLib.h>
-#include <Library/BaseMemoryLib.h>
-
-EFI_MMC_HOST_PROTOCOL     *gpMmcHost;
-
-// Untested ...
-//#define USE_STREAM
-
-#define MMCI0_BLOCKLEN 512
-#define MMCI0_POW2_BLOCKLEN     9
-#define MMCI0_TIMEOUT           1000
-
-#define SYS_MCI_CARDIN          BIT0
-#define SYS_MCI_WPROT           BIT1
-
-BOOLEAN
-MciIsPowerOn (
-  VOID
-  )
-{
-  return ((MmioRead32 (MCI_POWER_CONTROL_REG) & MCI_POWER_ON) == MCI_POWER_ON);
-}
-
-EFI_STATUS
-MciInitialize (
-  VOID
-  )
-{
-  MCI_TRACE ("MciInitialize()");
-  return EFI_SUCCESS;
-}
-
-BOOLEAN
-MciIsCardPresent (
-  IN EFI_MMC_HOST_PROTOCOL     *This
-  )
-{
-  return (MmioRead32 (FixedPcdGet32 (PcdPL180SysMciRegAddress)) & SYS_MCI_CARDIN);
-}
-
-BOOLEAN
-MciIsReadOnly (
-  IN EFI_MMC_HOST_PROTOCOL     *This
-  )
-{
-  return (MmioRead32 (FixedPcdGet32 (PcdPL180SysMciRegAddress)) & SYS_MCI_WPROT);
-}
-
-// Convert block size to 2^n
-STATIC
-UINT32
-GetPow2BlockLen (
-  IN UINT32 BlockLen
-  )
-{
-  UINTN Loop;
-  UINTN Pow2BlockLen;
-
-  Loop    = 0x8000;
-  Pow2BlockLen = 15;
-  do {
-    Loop = (Loop >> 1) & 0xFFFF;
-    Pow2BlockLen--;
-  } while (Pow2BlockLen && (!(Loop & BlockLen)));
-
-  return Pow2BlockLen;
-}
-
-VOID
-MciPrepareDataPath (
-  IN UINTN TransferDirection
-  )
-{
-  // Set Data Length & Data Timer
-  MmioWrite32 (MCI_DATA_TIMER_REG, 0xFFFFFFF);
-  MmioWrite32 (MCI_DATA_LENGTH_REG, MMCI0_BLOCKLEN);
-
-#ifndef USE_STREAM
-  //Note: we are using a hardcoded BlockLen (==512). If we decide to use a variable size, we could
-  // compute the pow2 of BlockLen with the above function GetPow2BlockLen ()
-  MmioWrite32 (MCI_DATA_CTL_REG, MCI_DATACTL_ENABLE | MCI_DATACTL_DMA_ENABLE | TransferDirection | (MMCI0_POW2_BLOCKLEN << 4));
-#else
-  MmioWrite32 (MCI_DATA_CTL_REG, MCI_DATACTL_ENABLE | MCI_DATACTL_DMA_ENABLE | TransferDirection | MCI_DATACTL_STREAM_TRANS);
-#endif
-}
-
-EFI_STATUS
-MciSendCommand (
-  IN EFI_MMC_HOST_PROTOCOL     *This,
-  IN MMC_CMD                    MmcCmd,
-  IN UINT32                     Argument
-  )
-{
-  UINT32  Status;
-  UINT32  Cmd;
-  UINTN   RetVal;
-  UINTN   CmdCtrlReg;
-  UINT32  DoneMask;
-
-  RetVal = EFI_SUCCESS;
-
-  if ((MmcCmd == MMC_CMD17) || (MmcCmd == MMC_CMD11)) {
-    MciPrepareDataPath (MCI_DATACTL_CARD_TO_CONT);
-  } else if ((MmcCmd == MMC_CMD24) || (MmcCmd == MMC_CMD20)) {
-    MciPrepareDataPath (MCI_DATACTL_CONT_TO_CARD);
-  } else if (MmcCmd == MMC_CMD6) {
-    MmioWrite32 (MCI_DATA_TIMER_REG, 0xFFFFFFF);
-    MmioWrite32 (MCI_DATA_LENGTH_REG, 64);
-#ifndef USE_STREAM
-    MmioWrite32 (MCI_DATA_CTL_REG, MCI_DATACTL_ENABLE | MCI_DATACTL_CARD_TO_CONT | GetPow2BlockLen (64));
-#else
-    MmioWrite32 (MCI_DATA_CTL_REG, MCI_DATACTL_ENABLE | MCI_DATACTL_CARD_TO_CONT | MCI_DATACTL_STREAM_TRANS);
-#endif
-  } else if (MmcCmd == MMC_ACMD51) {
-    MmioWrite32 (MCI_DATA_TIMER_REG, 0xFFFFFFF);
-    /* SCR register is 8 bytes long. */
-    MmioWrite32 (MCI_DATA_LENGTH_REG, 8);
-#ifndef USE_STREAM
-    MmioWrite32 (MCI_DATA_CTL_REG, MCI_DATACTL_ENABLE | MCI_DATACTL_CARD_TO_CONT | GetPow2BlockLen (8));
-#else
-    MmioWrite32 (MCI_DATA_CTL_REG, MCI_DATACTL_ENABLE | MCI_DATACTL_CARD_TO_CONT | MCI_DATACTL_STREAM_TRANS);
-#endif
-  }
-
-  // Create Command for PL180
-  Cmd = (MMC_GET_INDX (MmcCmd) & INDX_MASK)  | MCI_CPSM_ENABLE;
-  if (MmcCmd & MMC_CMD_WAIT_RESPONSE) {
-    Cmd |= MCI_CPSM_WAIT_RESPONSE;
-  }
-
-  if (MmcCmd & MMC_CMD_LONG_RESPONSE) {
-    Cmd |= MCI_CPSM_LONG_RESPONSE;
-  }
-
-  // Clear Status register static flags
-  MmioWrite32 (MCI_CLEAR_STATUS_REG, MCI_CLR_ALL_STATUS);
-
-  // Write to command argument register
-  MmioWrite32 (MCI_ARGUMENT_REG, Argument);
-
-  // Write to command register
-  MmioWrite32 (MCI_COMMAND_REG, Cmd);
-
-  DoneMask  = (Cmd & MCI_CPSM_WAIT_RESPONSE)
-                ? (MCI_STATUS_CMD_RESPEND | MCI_STATUS_CMD_ERROR)
-                : (MCI_STATUS_CMD_SENT    | MCI_STATUS_CMD_ERROR);
-  do {
-    Status = MmioRead32 (MCI_STATUS_REG);
-  } while (! (Status & DoneMask));
-
-  if ((Status & MCI_STATUS_CMD_ERROR)) {
-    // Clear Status register error flags
-    MmioWrite32 (MCI_CLEAR_STATUS_REG, MCI_STATUS_CMD_ERROR);
-
-    if ((Status & MCI_STATUS_CMD_START_BIT_ERROR)) {
-      DEBUG ((EFI_D_ERROR, "MciSendCommand(CmdIndex:%d) Start bit Error! Response:0x%X Status:0x%x\n", (Cmd & 0x3F), MmioRead32 (MCI_RESPONSE0_REG), Status));
-      RetVal = EFI_NO_RESPONSE;
-    } else if ((Status & MCI_STATUS_CMD_CMDTIMEOUT)) {
-      //DEBUG ((EFI_D_ERROR, "MciSendCommand(CmdIndex:%d) TIMEOUT! Response:0x%X Status:0x%x\n", (Cmd & 0x3F), MmioRead32 (MCI_RESPONSE0_REG), Status));
-      RetVal = EFI_TIMEOUT;
-    } else if ((!(MmcCmd & MMC_CMD_NO_CRC_RESPONSE)) && (Status & MCI_STATUS_CMD_CMDCRCFAIL)) {
-      // The CMD1 and response type R3 do not contain CRC. We should ignore the CRC failed Status.
-      RetVal = EFI_CRC_ERROR;
-    }
-  }
-
-  // Disable Command Path
-  CmdCtrlReg = MmioRead32 (MCI_COMMAND_REG);
-  MmioWrite32 (MCI_COMMAND_REG, (CmdCtrlReg & ~MCI_CPSM_ENABLE));
-  return RetVal;
-}
-
-EFI_STATUS
-MciReceiveResponse (
-  IN EFI_MMC_HOST_PROTOCOL     *This,
-  IN MMC_RESPONSE_TYPE          Type,
-  IN UINT32*                    Buffer
-  )
-{
-  if (Buffer == NULL) {
-    return EFI_INVALID_PARAMETER;
-  }
-
-  if (   (Type == MMC_RESPONSE_TYPE_R1)
-      || (Type == MMC_RESPONSE_TYPE_R1b)
-      || (Type == MMC_RESPONSE_TYPE_R3)
-      || (Type == MMC_RESPONSE_TYPE_R6)
-      || (Type == MMC_RESPONSE_TYPE_R7))
-  {
-    Buffer[0] = MmioRead32 (MCI_RESPONSE3_REG);
-  } else if (Type == MMC_RESPONSE_TYPE_R2) {
-    Buffer[0] = MmioRead32 (MCI_RESPONSE0_REG);
-    Buffer[1] = MmioRead32 (MCI_RESPONSE1_REG);
-    Buffer[2] = MmioRead32 (MCI_RESPONSE2_REG);
-    Buffer[3] = MmioRead32 (MCI_RESPONSE3_REG);
-  }
-
-  return EFI_SUCCESS;
-}
-
-EFI_STATUS
-MciReadBlockData (
-  IN EFI_MMC_HOST_PROTOCOL     *This,
-  IN EFI_LBA                    Lba,
-  IN UINTN                      Length,
-  IN UINT32*                    Buffer
-  )
-{
-  UINTN Loop;
-  UINTN Finish;
-  UINTN Status;
-  EFI_STATUS RetVal;
-  UINTN  DataCtrlReg;
-  EFI_TPL Tpl;
-
-  RetVal = EFI_SUCCESS;
-
-  // Read data from the RX FIFO
-  Loop   = 0;
-  if (Length < MMCI0_BLOCKLEN) {
-    Finish = Length / 4;
-  } else {
-    Finish = MMCI0_BLOCKLEN / 4;
-  }
-
-  // Raise the TPL at the highest level to disable Interrupts.
-  Tpl = gBS->RaiseTPL (TPL_HIGH_LEVEL);
-
-  do {
-    // Read the Status flags
-    Status = MmioRead32 (MCI_STATUS_REG);
-
-    // Do eight reads if possible else a single read
-    if (Status & MCI_STATUS_CMD_RXFIFOHALFFULL) {
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-    } else if (Status & MCI_STATUS_CMD_RXDATAAVAILBL) {
-      Buffer[Loop] = MmioRead32(MCI_FIFO_REG);
-      Loop++;
-    } else {
-      //Check for error conditions and timeouts
-      if (Status & MCI_STATUS_CMD_DATATIMEOUT) {
-        DEBUG ((EFI_D_ERROR, "MciReadBlockData(): TIMEOUT! Response:0x%X Status:0x%x\n", MmioRead32 (MCI_RESPONSE0_REG), Status));
-        RetVal = EFI_TIMEOUT;
-        break;
-      } else if (Status & MCI_STATUS_CMD_DATACRCFAIL) {
-        DEBUG ((EFI_D_ERROR, "MciReadBlockData(): CRC Error! Response:0x%X Status:0x%x\n", MmioRead32 (MCI_RESPONSE0_REG), Status));
-        RetVal = EFI_CRC_ERROR;
-        break;
-      } else if (Status & MCI_STATUS_CMD_START_BIT_ERROR) {
-        DEBUG ((EFI_D_ERROR, "MciReadBlockData(): Start-bit Error! Response:0x%X Status:0x%x\n", MmioRead32 (MCI_RESPONSE0_REG), Status));
-        RetVal = EFI_NO_RESPONSE;
-        break;
-      }
-    }
-    //clear RX over run flag
-    if(Status & MCI_STATUS_CMD_RXOVERRUN) {
-      MmioWrite32(MCI_CLEAR_STATUS_REG, MCI_STATUS_CMD_RXOVERRUN);
-    }
-  } while ((Loop < Finish));
-
-  // Restore Tpl
-  gBS->RestoreTPL (Tpl);
-
-  // Clear Status flags
-  MmioWrite32 (MCI_CLEAR_STATUS_REG, MCI_CLR_ALL_STATUS);
-
-  //Disable Data path
-  DataCtrlReg = MmioRead32 (MCI_DATA_CTL_REG);
-  MmioWrite32 (MCI_DATA_CTL_REG, (DataCtrlReg & MCI_DATACTL_DISABLE_MASK));
-
-  return RetVal;
-}
-
-EFI_STATUS
-MciWriteBlockData (
-  IN EFI_MMC_HOST_PROTOCOL     *This,
-  IN EFI_LBA                   Lba,
-  IN UINTN                     Length,
-  IN UINT32*                   Buffer
-  )
-{
-  UINTN Loop;
-  UINTN Finish;
-  UINTN Timer;
-  UINTN Status;
-  EFI_STATUS RetVal;
-  UINTN  DataCtrlReg;
-  EFI_TPL Tpl;
-
-  RetVal = EFI_SUCCESS;
-
-  // Write the data to the TX FIFO
-  Loop   = 0;
-  Finish = MMCI0_BLOCKLEN / 4;
-  Timer  = MMCI0_TIMEOUT * 100;
-
-  // Raise the TPL at the highest level to disable Interrupts.
-  Tpl = gBS->RaiseTPL (TPL_HIGH_LEVEL);
-
-  do {
-    // Read the Status flags
-    Status = MmioRead32 (MCI_STATUS_REG);
-
-    // Do eight writes if possible else a single write
-    if (Status & MCI_STATUS_CMD_TXFIFOHALFEMPTY) {
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-      MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-      Loop++;
-    } else if (!(Status & MCI_STATUS_CMD_TXFIFOFULL)) {
-        MmioWrite32(MCI_FIFO_REG, Buffer[Loop]);
-        Loop++;
-    } else {
-      // Check for error conditions and timeouts
-      if (Status & MCI_STATUS_CMD_DATATIMEOUT) {
-        DEBUG ((EFI_D_ERROR, "MciWriteBlockData(): TIMEOUT! Response:0x%X Status:0x%x\n", MmioRead32 (MCI_RESPONSE0_REG), Status));
-        RetVal = EFI_TIMEOUT;
-        goto Exit;
-      } else if (Status & MCI_STATUS_CMD_DATACRCFAIL) {
-        DEBUG ((EFI_D_ERROR, "MciWriteBlockData(): CRC Error! Response:0x%X Status:0x%x\n", MmioRead32 (MCI_RESPONSE0_REG), Status));
-        RetVal = EFI_CRC_ERROR;
-        goto Exit;
-      } else if (Status & MCI_STATUS_CMD_TX_UNDERRUN) {
-        DEBUG ((EFI_D_ERROR, "MciWriteBlockData(): TX buffer Underrun! Response:0x%X Status:0x%x, Number of bytes written 0x%x\n",MmioRead32(MCI_RESPONSE0_REG),Status, Loop));
-        RetVal = EFI_BUFFER_TOO_SMALL;
-        ASSERT(0);
-        goto Exit;
-      }
-    }
-  } while (Loop < Finish);
-
-  // Restore Tpl
-  gBS->RestoreTPL (Tpl);
-
-  // Wait for FIFO to drain
-  Timer  = MMCI0_TIMEOUT * 60;
-  Status = MmioRead32 (MCI_STATUS_REG);
-#ifndef USE_STREAM
-  // Single block
-  while (((Status & MCI_STATUS_TXDONE) != MCI_STATUS_TXDONE) && Timer) {
-#else
-  // Stream
-  while (((Status & MCI_STATUS_CMD_DATAEND) != MCI_STATUS_CMD_DATAEND) && Timer) {
-#endif
-    NanoSecondDelay(10);
-    Status = MmioRead32 (MCI_STATUS_REG);
-    Timer--;
-  }
-
-  // Clear Status flags
-  MmioWrite32 (MCI_CLEAR_STATUS_REG, MCI_CLR_ALL_STATUS);
-
-  if (Timer == 0) {
-    DEBUG ((EFI_D_ERROR, "MciWriteBlockData(): Data End timeout Number of words written 0x%x\n", Loop));
-    RetVal = EFI_TIMEOUT;
-  }
-
-Exit:
-  // Disable Data path
-  DataCtrlReg = MmioRead32 (MCI_DATA_CTL_REG);
-  MmioWrite32 (MCI_DATA_CTL_REG, (DataCtrlReg & MCI_DATACTL_DISABLE_MASK));
-  return RetVal;
-}
-
-EFI_STATUS
-MciNotifyState (
-  IN  EFI_MMC_HOST_PROTOCOL     *This,
-  IN MMC_STATE                  State
-  )
-{
-  UINT32      Data32;
-
-  switch (State) {
-  case MmcInvalidState:
-    ASSERT (0);
-    break;
-  case MmcHwInitializationState:
-    // If device already turn on then restart it
-    Data32 = MmioRead32 (MCI_POWER_CONTROL_REG);
-    if ((Data32 & 0x2) == MCI_POWER_UP) {
-      MCI_TRACE ("MciNotifyState(MmcHwInitializationState): TurnOff MCI");
-
-      // Turn off
-      MmioWrite32 (MCI_CLOCK_CONTROL_REG, 0);
-      MmioWrite32 (MCI_POWER_CONTROL_REG, 0);
-      MicroSecondDelay (100);
-    }
-
-    MCI_TRACE ("MciNotifyState(MmcHwInitializationState): TurnOn MCI");
-    // Setup clock
-    //  - 0x1D = 29 => should be the clock divider to be less than 400kHz at MCLK = 24Mhz
-    MmioWrite32 (MCI_CLOCK_CONTROL_REG, 0x1D | MCI_CLOCK_ENABLE | MCI_CLOCK_POWERSAVE);
-
-    // Set the voltage
-    MmioWrite32 (MCI_POWER_CONTROL_REG, MCI_POWER_OPENDRAIN | (15<<2));
-    MmioWrite32 (MCI_POWER_CONTROL_REG, MCI_POWER_ROD | MCI_POWER_OPENDRAIN | (15<<2) | MCI_POWER_UP);
-    MicroSecondDelay (10);
-    MmioWrite32 (MCI_POWER_CONTROL_REG, MCI_POWER_ROD | MCI_POWER_OPENDRAIN | (15<<2) | MCI_POWER_ON);
-    MicroSecondDelay (100);
-
-    // Set Data Length & Data Timer
-    MmioWrite32 (MCI_DATA_TIMER_REG, 0xFFFFF);
-    MmioWrite32 (MCI_DATA_LENGTH_REG, 8);
-
-    ASSERT ((MmioRead32 (MCI_POWER_CONTROL_REG) & 0x3) == MCI_POWER_ON);
-    break;
-  case MmcIdleState:
-    MCI_TRACE ("MciNotifyState(MmcIdleState)");
-    break;
-  case MmcReadyState:
-    MCI_TRACE ("MciNotifyState(MmcReadyState)");
-    break;
-  case MmcIdentificationState:
-    MCI_TRACE ("MciNotifyState (MmcIdentificationState)");
-    break;
-  case MmcStandByState:{
-    volatile UINT32 PwrCtrlReg;
-    MCI_TRACE ("MciNotifyState (MmcStandByState)");
-
-    // Enable MCICMD push-pull drive
-    PwrCtrlReg = MmioRead32 (MCI_POWER_CONTROL_REG);
-    //Disable Open Drain output
-    PwrCtrlReg &= ~ (MCI_POWER_OPENDRAIN);
-    MmioWrite32 (MCI_POWER_CONTROL_REG, PwrCtrlReg);
-
-    // Set MMCI0 clock to 4MHz (24MHz may be possible with cache enabled)
-    //
-    // Note: Increasing clock speed causes TX FIFO under-run errors.
-    //       So careful when optimising this driver for higher performance.
-    //
-    MmioWrite32(MCI_CLOCK_CONTROL_REG,0x02 | MCI_CLOCK_ENABLE | MCI_CLOCK_POWERSAVE);
-    // Set MMCI0 clock to 24MHz (by bypassing the divider)
-    //MmioWrite32(MCI_CLOCK_CONTROL_REG,MCI_CLOCK_BYPASS | MCI_CLOCK_ENABLE);
-    break;
-  }
-  case MmcTransferState:
-    //MCI_TRACE ("MciNotifyState(MmcTransferState)");
-    break;
-  case MmcSendingDataState:
-    MCI_TRACE ("MciNotifyState(MmcSendingDataState)");
-    break;
-  case MmcReceiveDataState:
-    MCI_TRACE ("MciNotifyState(MmcReceiveDataState)");
-    break;
-  case MmcProgrammingState:
-    MCI_TRACE ("MciNotifyState(MmcProgrammingState)");
-    break;
-  case MmcDisconnectState:
-    MCI_TRACE ("MciNotifyState(MmcDisconnectState)");
-    break;
-  default:
-    ASSERT (0);
-  }
-  return EFI_SUCCESS;
-}
-
-EFI_GUID mPL180MciDevicePathGuid = EFI_CALLER_ID_GUID;
-
-EFI_STATUS
-MciBuildDevicePath (
-  IN EFI_MMC_HOST_PROTOCOL      *This,
-  IN EFI_DEVICE_PATH_PROTOCOL   **DevicePath
-  )
-{
-  EFI_DEVICE_PATH_PROTOCOL    *NewDevicePathNode;
-
-  NewDevicePathNode = CreateDeviceNode (HARDWARE_DEVICE_PATH, HW_VENDOR_DP, sizeof (VENDOR_DEVICE_PATH));
-  CopyGuid (& ((VENDOR_DEVICE_PATH*)NewDevicePathNode)->Guid, &mPL180MciDevicePathGuid);
-
-  *DevicePath = NewDevicePathNode;
-  return EFI_SUCCESS;
-}
-
-EFI_MMC_HOST_PROTOCOL gMciHost = {
-  MMC_HOST_PROTOCOL_REVISION,
-  MciIsCardPresent,
-  MciIsReadOnly,
-  MciBuildDevicePath,
-  MciNotifyState,
-  MciSendCommand,
-  MciReceiveResponse,
-  MciReadBlockData,
-  MciWriteBlockData
-};
-
-EFI_STATUS
-PL180MciDxeInitialize (
-  IN EFI_HANDLE         ImageHandle,
-  IN EFI_SYSTEM_TABLE   *SystemTable
-  )
-{
-  EFI_STATUS    Status;
-  EFI_HANDLE    Handle;
-
-  DEBUG ((EFI_D_WARN, "Probing ID registers at 0x%lx for a PL180\n",
-    MCI_PERIPH_ID_REG0));
-
-  // Check if this is a PL180
-  if (MmioRead8 (MCI_PERIPH_ID_REG0) != MCI_PERIPH_ID0 ||
-      MmioRead8 (MCI_PERIPH_ID_REG1) != MCI_PERIPH_ID1 ||
-      MmioRead8 (MCI_PERIPH_ID_REG2) != MCI_PERIPH_ID2 ||
-      MmioRead8 (MCI_PCELL_ID_REG0)  != MCI_PCELL_ID0  ||
-      MmioRead8 (MCI_PCELL_ID_REG1)  != MCI_PCELL_ID1  ||
-      MmioRead8 (MCI_PCELL_ID_REG2)  != MCI_PCELL_ID2  ||
-      MmioRead8 (MCI_PCELL_ID_REG3)  != MCI_PCELL_ID3) {
-
-    DEBUG ((EFI_D_WARN, "Probing ID registers at 0x%lx for a PL180"
-      " failed\n", MCI_PERIPH_ID_REG0));
-    return EFI_NOT_FOUND;
-  }
-
-  Handle = NULL;
-
-  MCI_TRACE ("PL180MciDxeInitialize()");
-
-  //Publish Component Name, BlockIO protocol interfaces
-  Status = gBS->InstallMultipleProtocolInterfaces (
-                  &Handle,
-                  &gEfiMmcHostProtocolGuid,         &gMciHost,
-                  NULL
-                  );
-  ASSERT_EFI_ERROR (Status);
-
-  return EFI_SUCCESS;
-}

ArmPlatformPkg/Drivers/PL180MciDxe/PL180Mci.h

@@ -1,162 +0,0 @@
-/** @file
-  Header for the MMC Host Protocol implementation for the ARM PrimeCell PL180.
-
-  Copyright (c) 2011-2012, ARM Limited. All rights reserved.
-
-  SPDX-License-Identifier: BSD-2-Clause-Patent
-
-**/
-
-#ifndef __PL180_MCI_H
-#define __PL180_MCI_H
-
-#include <Uefi.h>
-
-#include <Protocol/MmcHost.h>
-
-#include <Library/UefiLib.h>
-#include <Library/DebugLib.h>
-#include <Library/UefiBootServicesTableLib.h>
-#include <Library/IoLib.h>
-#include <Library/TimerLib.h>
-#include <Library/PcdLib.h>
-
-#define PL180_MCI_DXE_VERSION           0x10
-
-#define MCI_SYSCTL  FixedPcdGet32 (PcdPL180MciBaseAddress)
-
-#define MCI_POWER_CONTROL_REG           (MCI_SYSCTL + 0x000)
-#define MCI_CLOCK_CONTROL_REG           (MCI_SYSCTL + 0x004)
-#define MCI_ARGUMENT_REG                (MCI_SYSCTL + 0x008)
-#define MCI_COMMAND_REG                 (MCI_SYSCTL + 0x00C)
-#define MCI_RESPCMD_REG                 (MCI_SYSCTL + 0x010)
-#define MCI_RESPONSE3_REG               (MCI_SYSCTL + 0x014)
-#define MCI_RESPONSE2_REG               (MCI_SYSCTL + 0x018)
-#define MCI_RESPONSE1_REG               (MCI_SYSCTL + 0x01C)
-#define MCI_RESPONSE0_REG               (MCI_SYSCTL + 0x020)
-#define MCI_DATA_TIMER_REG              (MCI_SYSCTL + 0x024)
-#define MCI_DATA_LENGTH_REG             (MCI_SYSCTL + 0x028)
-#define MCI_DATA_CTL_REG                (MCI_SYSCTL + 0x02C)
-#define MCI_DATA_COUNTER                (MCI_SYSCTL + 0x030)
-#define MCI_STATUS_REG                  (MCI_SYSCTL + 0x034)
-#define MCI_CLEAR_STATUS_REG            (MCI_SYSCTL + 0x038)
-#define MCI_INT0_MASK_REG               (MCI_SYSCTL + 0x03C)
-#define MCI_INT1_MASK_REG               (MCI_SYSCTL + 0x040)
-#define MCI_SELECT_REG                  (MCI_SYSCTL + 0x044)
-#define MCI_FIFOCOUNT_REG               (MCI_SYSCTL + 0x048)
-#define MCI_FIFO_REG                    (MCI_SYSCTL + 0x080)
-#define MCI_PERIPH_ID_REG0              (MCI_SYSCTL + 0xFE0)
-#define MCI_PERIPH_ID_REG1              (MCI_SYSCTL + 0xFE4)
-#define MCI_PERIPH_ID_REG2              (MCI_SYSCTL + 0xFE8)
-#define MCI_PERIPH_ID_REG3              (MCI_SYSCTL + 0xFEC)
-#define MCI_PCELL_ID_REG0               (MCI_SYSCTL + 0xFF0)
-#define MCI_PCELL_ID_REG1               (MCI_SYSCTL + 0xFF4)
-#define MCI_PCELL_ID_REG2               (MCI_SYSCTL + 0xFF8)
-#define MCI_PCELL_ID_REG3               (MCI_SYSCTL + 0xFFC)
-
-#define MCI_PERIPH_ID0                  0x80
-#define MCI_PERIPH_ID1                  0x11
-#define MCI_PERIPH_ID2                  0x04
-#define MCI_PERIPH_ID3                  0x00
-#define MCI_PCELL_ID0                   0x0D
-#define MCI_PCELL_ID1                   0xF0
-#define MCI_PCELL_ID2                   0x05
-#define MCI_PCELL_ID3                   0xB1
-
-#define MCI_POWER_OFF                   0
-#define MCI_POWER_UP                    BIT1
-#define MCI_POWER_ON                    (BIT1 | BIT0)
-#define MCI_POWER_OPENDRAIN             BIT6
-#define MCI_POWER_ROD                   BIT7
-
-#define MCI_CLOCK_ENABLE                BIT8
-#define MCI_CLOCK_POWERSAVE             BIT9
-#define MCI_CLOCK_BYPASS                BIT10
-#define MCI_CLOCK_WIDEBUS               BIT11
-
-#define MCI_STATUS_CMD_CMDCRCFAIL       BIT0
-#define MCI_STATUS_CMD_DATACRCFAIL      BIT1
-#define MCI_STATUS_CMD_CMDTIMEOUT       BIT2
-#define MCI_STATUS_CMD_DATATIMEOUT      BIT3
-#define MCI_STATUS_CMD_TX_UNDERRUN      BIT4
-#define MCI_STATUS_CMD_RXOVERRUN        BIT5
-#define MCI_STATUS_CMD_RESPEND          BIT6
-#define MCI_STATUS_CMD_SENT             BIT7
-#define MCI_STATUS_CMD_DATAEND          BIT8
-#define MCI_STATUS_CMD_START_BIT_ERROR  BIT9
-#define MCI_STATUS_CMD_DATABLOCKEND     BIT10
-#define MCI_STATUS_CMD_ACTIVE           BIT11
-#define MCI_STATUS_CMD_TXACTIVE         BIT12
-#define MCI_STATUS_CMD_RXACTIVE         BIT13
-#define MCI_STATUS_CMD_TXFIFOHALFEMPTY  BIT14
-#define MCI_STATUS_CMD_RXFIFOHALFFULL   BIT15
-#define MCI_STATUS_CMD_TXFIFOFULL       BIT16
-#define MCI_STATUS_CMD_RXFIFOFULL       BIT17
-#define MCI_STATUS_CMD_TXFIFOEMPTY      BIT18
-#define MCI_STATUS_CMD_RXFIFOEMPTY      BIT19
-#define MCI_STATUS_CMD_TXDATAAVAILBL    BIT20
-#define MCI_STATUS_CMD_RXDATAAVAILBL    BIT21
-
-#define MCI_STATUS_TXDONE               (MCI_STATUS_CMD_DATAEND | MCI_STATUS_CMD_DATABLOCKEND)
-#define MCI_STATUS_RXDONE               (MCI_STATUS_CMD_DATAEND | MCI_STATUS_CMD_DATABLOCKEND)
-#define MCI_STATUS_READ_ERROR           (  MCI_STATUS_CMD_DATACRCFAIL     \
-                                         | MCI_STATUS_CMD_DATATIMEOUT     \
-                                         | MCI_STATUS_CMD_RXOVERRUN       \
-                                         | MCI_STATUS_CMD_START_BIT_ERROR )
-#define MCI_STATUS_WRITE_ERROR          (  MCI_STATUS_CMD_DATACRCFAIL \
-                                         | MCI_STATUS_CMD_DATATIMEOUT \
-                                         | MCI_STATUS_CMD_TX_UNDERRUN )
-#define MCI_STATUS_CMD_ERROR            (  MCI_STATUS_CMD_CMDCRCFAIL      \
-                                         | MCI_STATUS_CMD_CMDTIMEOUT      \
-                                         | MCI_STATUS_CMD_START_BIT_ERROR )
-
-#define MCI_CLR_CMD_STATUS              (  MCI_STATUS_CMD_RESPEND \
-                                         | MCI_STATUS_CMD_SENT    \
-                                         | MCI_STATUS_CMD_ERROR )
-
-#define MCI_CLR_READ_STATUS             (  MCI_STATUS_RXDONE     \
-                                         | MCI_STATUS_READ_ERROR )
-
-#define MCI_CLR_WRITE_STATUS            (  MCI_STATUS_TXDONE      \
-                                         | MCI_STATUS_WRITE_ERROR )
-
-#define MCI_CLR_ALL_STATUS              (BIT11 - 1)
-
-#define MCI_DATACTL_DISABLE_MASK        0xFE
-#define MCI_DATACTL_ENABLE              BIT0
-#define MCI_DATACTL_CONT_TO_CARD        0
-#define MCI_DATACTL_CARD_TO_CONT        BIT1
-#define MCI_DATACTL_BLOCK_TRANS         0
-#define MCI_DATACTL_STREAM_TRANS        BIT2
-#define MCI_DATACTL_DMA_DISABLED        0
-#define MCI_DATACTL_DMA_ENABLE          BIT3
-
-#define INDX_MASK                       0x3F
-
-#define MCI_CPSM_WAIT_RESPONSE          BIT6
-#define MCI_CPSM_LONG_RESPONSE          BIT7
-#define MCI_CPSM_LONG_INTERRUPT         BIT8
-#define MCI_CPSM_LONG_PENDING           BIT9
-#define MCI_CPSM_ENABLE                 BIT10
-
-#define MCI_TRACE(txt)                  DEBUG ((EFI_D_BLKIO, "ARM_MCI: " txt "\n"))
-
-EFI_STATUS
-EFIAPI
-MciGetDriverName (
-  IN  EFI_COMPONENT_NAME_PROTOCOL  *This,
-  IN  CHAR8                        *Language,
-  OUT CHAR16                       **DriverName
-  );
-
-EFI_STATUS
-EFIAPI
-MciGetControllerName (
-  IN  EFI_COMPONENT_NAME_PROTOCOL                     *This,
-  IN  EFI_HANDLE                                      ControllerHandle,
-  IN  EFI_HANDLE                                      ChildHandle        OPTIONAL,
-  IN  CHAR8                                           *Language,
-  OUT CHAR16                                          **ControllerName
-  );
-
-#endif

ArmPlatformPkg/Drivers/PL180MciDxe/PL180MciDxe.inf

@@ -1,46 +0,0 @@
-#/** @file
-#  INF file for the MMC Host Protocol implementation for the ARM PrimeCell PL180.
-#
-#  Copyright (c) 2011, ARM Limited. All rights reserved.
-#
-#  SPDX-License-Identifier: BSD-2-Clause-Patent
-#
-#**/
-
-[Defines]
-  INF_VERSION                    = 0x00010005
-  BASE_NAME                      = PL180MciDxe
-  FILE_GUID                      = 09831032-6fa3-4484-af4f-0a000a8d3a82
-  MODULE_TYPE                    = DXE_DRIVER
-  VERSION_STRING                 = 1.0
-
-  ENTRY_POINT                    = PL180MciDxeInitialize
-
-[Sources.common]
-  PL180Mci.c
-
-[Packages]
-  ArmPlatformPkg/ArmPlatformPkg.dec
-  EmbeddedPkg/EmbeddedPkg.dec
-  MdePkg/MdePkg.dec
-
-[LibraryClasses]
-  BaseLib
-  UefiLib
-  UefiDriverEntryPoint
-  BaseMemoryLib
-  ArmLib
-  IoLib
-  TimerLib
-
-[Protocols]
-  gEfiCpuArchProtocolGuid
-  gEfiDevicePathProtocolGuid
-  gEfiMmcHostProtocolGuid
-
-[Pcd]
-  gArmPlatformTokenSpaceGuid.PcdPL180SysMciRegAddress
-  gArmPlatformTokenSpaceGuid.PcdPL180MciBaseAddress
-
-[Depex]
-  gEfiCpuArchProtocolGuid

ArmPlatformPkg/PrePeiCore/PrePeiCore.c

@@ -8,6 +8,7 @@
 **/
 
 #include <Library/BaseLib.h>
+#include <Library/CacheMaintenanceLib.h>
 #include <Library/DebugAgentLib.h>
 #include <Library/ArmLib.h>
 
@@ -59,13 +60,14 @@ CEntryPoint (
 {
   // Data Cache enabled on Primary core when MMU is enabled.
   ArmDisableDataCache ();
-  // Invalidate Data cache
-  ArmInvalidateDataCache ();
   // Invalidate instruction cache
   ArmInvalidateInstructionCache ();
   // Enable Instruction Caches on all cores.
   ArmEnableInstructionCache ();
 
+  InvalidateDataCacheRange ((VOID *)(UINTN)PcdGet64 (PcdCPUCoresStackBase),
+                            PcdGet32 (PcdCPUCorePrimaryStackSize));
+
   //
   // Note: Doesn't have to Enable CPU interface in non-secure world,
   // as Non-secure interface is already enabled in Secure world.

ArmPlatformPkg/PrePeiCore/PrePeiCoreMPCore.inf

@@ -44,6 +44,7 @@
 [LibraryClasses]
   ArmLib
   ArmPlatformLib
+  CacheMaintenanceLib
   BaseLib
   DebugLib
   DebugAgentLib

ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf

@@ -44,6 +44,7 @@
 [LibraryClasses]
   ArmLib
   ArmPlatformLib
+  CacheMaintenanceLib
   BaseLib
   DebugLib
   DebugAgentLib

ArmPlatformPkg/PrePi/PeiMPCore.inf

@@ -37,6 +37,7 @@
 
 [LibraryClasses]
   BaseLib
+  CacheMaintenanceLib
   DebugLib
   DebugAgentLib
   ArmLib

ArmPlatformPkg/PrePi/PeiUniCore.inf

@@ -37,6 +37,7 @@
 
 [LibraryClasses]
   BaseLib
+  CacheMaintenanceLib
   DebugLib
   DebugAgentLib
   ArmLib

ArmPlatformPkg/PrePi/PrePi.c

@@ -8,6 +8,7 @@
 
 #include <PiPei.h>
 
+#include <Library/CacheMaintenanceLib.h>
 #include <Library/DebugAgentLib.h>
 #include <Library/PrePiLib.h>
 #include <Library/PrintLib.h>
@@ -22,7 +23,7 @@
 #include "PrePi.h"
 
 #define IS_XIP() (((UINT64)FixedPcdGet64 (PcdFdBaseAddress) > mSystemMemoryEnd) || \
-                  ((FixedPcdGet64 (PcdFdBaseAddress) + FixedPcdGet32 (PcdFdSize)) < FixedPcdGet64 (PcdSystemMemoryBase)))
+                  ((FixedPcdGet64 (PcdFdBaseAddress) + FixedPcdGet32 (PcdFdSize)) <= FixedPcdGet64 (PcdSystemMemoryBase)))
 
 UINT64 mSystemMemoryEnd = FixedPcdGet64(PcdSystemMemoryBase) +
                           FixedPcdGet64(PcdSystemMemorySize) - 1;
@@ -178,8 +179,6 @@ CEntryPoint (
 
   // Data Cache enabled on Primary core when MMU is enabled.
   ArmDisableDataCache ();
-  // Invalidate Data cache
-  ArmInvalidateDataCache ();
   // Invalidate instruction cache
   ArmInvalidateInstructionCache ();
   // Enable Instruction Caches on all cores.
@@ -200,6 +199,10 @@ CEntryPoint (
 
   // If not primary Jump to Secondary Main
   if (ArmPlatformIsPrimaryCore (MpId)) {
+
+    InvalidateDataCacheRange ((VOID *)UefiMemoryBase,
+                              FixedPcdGet32 (PcdSystemMemoryUefiRegionSize));
+
     // Goto primary Main.
     PrimaryMain (UefiMemoryBase, StacksBase, StartTimeStamp);
   } else {
@@ -209,4 +212,3 @@ CEntryPoint (
   // DXE Core should always load and never return
   ASSERT (FALSE);
 }
-

ArmPlatformPkg/Scripts/Ds5/profile.py

@@ -1,328 +1,328 @@
-#!/usr/bin/python
-
-#
-#  Copyright (c) 2014, ARM Limited. All rights reserved.
-#
-#  SPDX-License-Identifier: BSD-2-Clause-Patent
-#
-
-import getopt
-import operator
-import os
-import pickle
-import sys
-from sys import argv
-from cStringIO import StringIO
-
-modules = {}
-functions = {}
-functions_addr = {}
-
-def usage():
-	print "-t,--trace: Location of the Trace file"
-	print "-s,--symbols: Location of the symbols and modules"
-
-def get_address_from_string(address):
-	return int(address.strip("S:").strip("N:").strip("EL2:").strip("EL1:"), 16)
-
-def get_module_from_addr(modules, addr):
-	for key,value in modules.items():
-		if (value['start'] <= addr) and (addr <= value['end']):
-			return key
-	return None
-
-def add_cycles_to_function(functions, func_name, addr, cycles):
-	if func_name != "<Unknown>":
-		# Check if we are still in the previous function
-		if add_cycles_to_function.prev_func_name == func_name:
-			add_cycles_to_function.prev_entry['cycles'] += cycles
-			return (add_cycles_to_function.prev_func_name, add_cycles_to_function.prev_module_name)
-
-		if func_name in functions.keys():
-			for module_name, module_value in functions[func_name].iteritems():
-				if (module_value['start'] <= addr) and (addr < module_value['end']):
-					module_value['cycles'] += cycles
-
-					add_cycles_to_function.prev_func_name   = func_name
-					add_cycles_to_function.prev_module_name = module_name
-					add_cycles_to_function.prev_entry       = module_value
-					return (func_name, module_name)
-				elif (module_value['end'] == 0):
-					module_value['cycles'] += cycles
-
-					add_cycles_to_function.prev_func_name   = func_name
-					add_cycles_to_function.prev_module_name = module_name
-					add_cycles_to_function.prev_entry       = module_value
-					return (func_name, module_name)
-
-		# Workaround to fix the 'info func' limitation that does not expose the 'static' function
-		module_name = get_module_from_addr(modules, addr)
-		functions[func_name] = {}
-		functions[func_name][module_name] = {}
-		functions[func_name][module_name]['start']  = 0
-		functions[func_name][module_name]['end']    = 0
-		functions[func_name][module_name]['cycles'] = cycles
-		functions[func_name][module_name]['count']  = 0
-
-		add_cycles_to_function.prev_func_name   = func_name
-		add_cycles_to_function.prev_module_name = module_name
-		add_cycles_to_function.prev_entry       = functions[func_name][module_name]
-		return (func_name, module_name)
-	else:
-		# Check if we are still in the previous function
-		if (add_cycles_to_function.prev_entry is not None) and (add_cycles_to_function.prev_entry['start'] <= addr) and (addr < add_cycles_to_function.prev_entry['end']):
-			add_cycles_to_function.prev_entry['cycles'] += cycles
-			return (add_cycles_to_function.prev_func_name, add_cycles_to_function.prev_module_name)
-
-		# Generate the key for the given address
-		key = addr & ~0x0FFF
-
-		if key not in functions_addr.keys():
-			if 'Unknown' not in functions.keys():
-				functions['Unknown'] = {}
-			if 'Unknown' not in functions['Unknown'].keys():
-				functions['Unknown']['Unknown'] = {}
-				functions['Unknown']['Unknown']['cycles'] = 0
-				functions['Unknown']['Unknown']['count'] = 0
-			functions['Unknown']['Unknown']['cycles'] += cycles
-
-			add_cycles_to_function.prev_func_name = None
-			return None
-
-		for func_key, module in functions_addr[key].iteritems():
-			for module_key, module_value in module.iteritems():
-				if (module_value['start'] <= addr) and (addr < module_value['end']):
-					module_value['cycles'] += cycles
-
-					# In case o <Unknown> we prefer to fallback on the direct search
-					add_cycles_to_function.prev_func_name   = func_key
-					add_cycles_to_function.prev_module_name = module_key
-					add_cycles_to_function.prev_entry       = module_value
-					return (func_key, module_key)
-
-	print "Warning: Function %s @ 0x%x not found" % (func_name, addr)
-
-	add_cycles_to_function.prev_func_name = None
-	return None
-
-# Static variables for the previous function
-add_cycles_to_function.prev_func_name = None
-add_cycles_to_function.prev_entry     = None
-
-def trace_read():
-	global trace_process
-	line = trace.readline()
-	trace_process += len(line)
-	return line
-
-#
-# Parse arguments
-#
-trace_name = None
-symbols_file = None
-
-opts,args = getopt.getopt(sys.argv[1:], "ht:vs:v", ["help","trace=","symbols="])
-if (opts is None) or (not opts):
-	usage()
-	sys.exit()
-
-for o,a in opts:
-    if o in ("-h","--help"):
-        usage()
-        sys.exit()
-    elif o in ("-t","--trace"):
-        trace_name = a
-    elif o in ("-s","--symbols"):
-        symbols_file = a
-    else:
-        assert False, "Unhandled option (%s)" % o
-
-#
-# We try first to see if we run the script from DS-5
-#
-try:
-	from arm_ds.debugger_v1 import Debugger
-	from arm_ds.debugger_v1 import DebugException
-
-	# Debugger object for accessing the debugger
-	debugger = Debugger()
-
-	# Initialisation commands
-	ec = debugger.getExecutionContext(0)
-	ec.getExecutionService().stop()
-	ec.getExecutionService().waitForStop()
-	# in case the execution context reference is out of date
-	ec = debugger.getExecutionContext(0)
-
-	#
-	# Get the module name and their memory range
-	#
-	info_file = ec.executeDSCommand("info file")
-	info_file_str = StringIO(info_file)
-
-	line = info_file_str.readline().strip('\n')
-	while line != '':
-		if ("Symbols from" in line):
-			# Get the module name from the line 'Symbols from "/home/...."'
-			module_name = line.split("\"")[1].split("/")[-1]
-			modules[module_name] = {}
-
-			# Look for the text section
-			line = info_file_str.readline().strip('\n')
-			while (line != '') and ("Symbols from" not in line):
-				if ("ER_RO" in line):
-					modules[module_name]['start'] = get_address_from_string(line.split()[0])
-					modules[module_name]['end']   = get_address_from_string(line.split()[2])
-					line = info_file_str.readline().strip('\n')
-					break;
-				if (".text" in line):
-					modules[module_name]['start'] = get_address_from_string(line.split()[0])
-					modules[module_name]['end']   = get_address_from_string(line.split()[2])
-					line = info_file_str.readline().strip('\n')
-					break;
-				line = info_file_str.readline().strip('\n')
-		line = info_file_str.readline().strip('\n')
-
-	#
-	# Get the function name and their memory range
-	#
-	info_func = ec.executeDSCommand("info func")
-	info_func_str = StringIO(info_func)
-
-	# Skip the first line 'Low-level symbols ...'
-	line = info_func_str.readline().strip('\n')
-	func_prev = None
-	while line != '':
-		# We ignore all the functions after 'Functions in'
-		if ("Functions in " in line):
-			line = info_func_str.readline().strip('\n')
-			while line != '':
-				line = info_func_str.readline().strip('\n')
-			line = info_func_str.readline().strip('\n')
-			continue
-
-		if ("Low-level symbols" in line):
-			# We need to fixup the last function of the module
-			if func_prev is not None:
-				func_prev['end'] = modules[module_name]['end']
-				func_prev = None
-
-			line = info_func_str.readline().strip('\n')
-			continue
-
-		func_name = line.split()[1]
-		func_start = get_address_from_string(line.split()[0])
-		module_name = get_module_from_addr(modules, func_start)
-
-		if func_name not in functions.keys():
-			functions[func_name] = {}
-		functions[func_name][module_name] = {}
-		functions[func_name][module_name]['start'] = func_start
-		functions[func_name][module_name]['cycles'] = 0
-		functions[func_name][module_name]['count'] = 0
-
-		# Set the end address of the previous function
-		if func_prev is not None:
-			func_prev['end'] = func_start
-		func_prev = functions[func_name][module_name]
-
-		line = info_func_str.readline().strip('\n')
-
-	# Fixup the last function
-	func_prev['end'] = modules[module_name]['end']
-
-	if symbols_file is not None:
-		pickle.dump((modules, functions), open(symbols_file, "w"))
-except:
-	if symbols_file is None:
-		print "Error: Symbols file is required when run out of ARM DS-5"
-		sys.exit()
-
-	(modules, functions) = pickle.load(open(symbols_file, "r"))
-
-#
-# Build optimized table for the <Unknown> functions
-#
-functions_addr = {}
-for func_key, module in functions.iteritems():
-	for module_key, module_value in module.iteritems():
-		key = module_value['start'] & ~0x0FFF
-		if key not in functions_addr.keys():
-			functions_addr[key] = {}
-		if func_key not in functions_addr[key].keys():
-			functions_addr[key][func_key] = {}
-		functions_addr[key][func_key][module_key] = module_value
-
-#
-# Process the trace file
-#
-if trace_name is None:
-	sys.exit()
-
-trace = open(trace_name, "r")
-trace_size = os.path.getsize(trace_name)
-trace_process = 0
-
-# Get the column names from the first line
-columns = trace_read().split()
-column_addr     = columns.index('Address')
-column_cycles   = columns.index('Cycles')
-column_function = columns.index('Function')
-
-line = trace_read()
-i = 0
-prev_callee = None
-while line:
-	try:
-		func_name = line.split('\t')[column_function].strip()
-		address   = get_address_from_string(line.split('\t')[column_addr])
-		cycles    = int(line.split('\t')[column_cycles])
-		callee = add_cycles_to_function(functions, func_name, address, cycles)
-		if (prev_callee != None) and (prev_callee != callee):
-			functions[prev_callee[0]][prev_callee[1]]['count'] += 1
-		prev_callee = callee
-	except ValueError:
-		pass
-	line = trace_read()
-	if ((i % 1000000) == 0) and (i != 0):
-		percent = (trace_process * 100.00) / trace_size
-		print "Processing file ... (%.2f %%)" % (percent)
-	i = i + 1
-
-# Fixup the last callee
-functions[prev_callee[0]][prev_callee[1]]['count'] += 1
-
-#
-# Process results
-#
-functions_cycles     = {}
-all_functions_cycles = {}
-total_cycles         = 0
-
-for func_key, module in functions.iteritems():
-	for module_key, module_value in module.iteritems():
-		key = "%s/%s" % (module_key, func_key)
-		functions_cycles[key] = (module_value['cycles'], module_value['count'])
-		total_cycles += module_value['cycles']
-
-		if func_key not in all_functions_cycles.keys():
-			all_functions_cycles[func_key] = (module_value['cycles'], module_value['count'])
-		else:
-			all_functions_cycles[func_key] = tuple(map(sum, zip(all_functions_cycles[func_key], (module_value['cycles'], module_value['count']))))
-
-sorted_functions_cycles     = sorted(functions_cycles.iteritems(), key=operator.itemgetter(1), reverse = True)
-sorted_all_functions_cycles = sorted(all_functions_cycles.items(), key=operator.itemgetter(1), reverse = True)
-
-print
-print "----"
-for (key,value) in sorted_functions_cycles[:20]:
-	if value[0] != 0:
-		print "%s (cycles: %d - %d%%, count: %d)" % (key, value[0], (value[0] * 100) / total_cycles, value[1])
-	else:
-		break;
-print "----"
-for (key,value) in sorted_all_functions_cycles[:20]:
-	if value[0] != 0:
-		print "%s (cycles: %d - %d%%, count: %d)" % (key, value[0], (value[0] * 100) / total_cycles, value[1])
-	else:
-		break;
+#!/usr/bin/python
+
+#
+#  Copyright (c) 2014, ARM Limited. All rights reserved.
+#
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import getopt
+import operator
+import os
+import pickle
+import sys
+from sys import argv
+from cStringIO import StringIO
+
+modules = {}
+functions = {}
+functions_addr = {}
+
+def usage():
+    print "-t,--trace: Location of the Trace file"
+    print "-s,--symbols: Location of the symbols and modules"
+
+def get_address_from_string(address):
+    return int(address.strip("S:").strip("N:").strip("EL2:").strip("EL1:"), 16)
+
+def get_module_from_addr(modules, addr):
+    for key,value in modules.items():
+        if (value['start'] <= addr) and (addr <= value['end']):
+            return key
+    return None
+
+def add_cycles_to_function(functions, func_name, addr, cycles):
+    if func_name != "<Unknown>":
+        # Check if we are still in the previous function
+        if add_cycles_to_function.prev_func_name == func_name:
+            add_cycles_to_function.prev_entry['cycles'] += cycles
+            return (add_cycles_to_function.prev_func_name, add_cycles_to_function.prev_module_name)
+
+        if func_name in functions.keys():
+            for module_name, module_value in functions[func_name].iteritems():
+                if (module_value['start'] <= addr) and (addr < module_value['end']):
+                    module_value['cycles'] += cycles
+
+                    add_cycles_to_function.prev_func_name   = func_name
+                    add_cycles_to_function.prev_module_name = module_name
+                    add_cycles_to_function.prev_entry       = module_value
+                    return (func_name, module_name)
+                elif (module_value['end'] == 0):
+                    module_value['cycles'] += cycles
+
+                    add_cycles_to_function.prev_func_name   = func_name
+                    add_cycles_to_function.prev_module_name = module_name
+                    add_cycles_to_function.prev_entry       = module_value
+                    return (func_name, module_name)
+
+        # Workaround to fix the 'info func' limitation that does not expose the 'static' function
+        module_name = get_module_from_addr(modules, addr)
+        functions[func_name] = {}
+        functions[func_name][module_name] = {}
+        functions[func_name][module_name]['start']  = 0
+        functions[func_name][module_name]['end']    = 0
+        functions[func_name][module_name]['cycles'] = cycles
+        functions[func_name][module_name]['count']  = 0
+
+        add_cycles_to_function.prev_func_name   = func_name
+        add_cycles_to_function.prev_module_name = module_name
+        add_cycles_to_function.prev_entry       = functions[func_name][module_name]
+        return (func_name, module_name)
+    else:
+        # Check if we are still in the previous function
+        if (add_cycles_to_function.prev_entry is not None) and (add_cycles_to_function.prev_entry['start'] <= addr) and (addr < add_cycles_to_function.prev_entry['end']):
+            add_cycles_to_function.prev_entry['cycles'] += cycles
+            return (add_cycles_to_function.prev_func_name, add_cycles_to_function.prev_module_name)
+
+        # Generate the key for the given address
+        key = addr & ~0x0FFF
+
+        if key not in functions_addr.keys():
+            if 'Unknown' not in functions.keys():
+                functions['Unknown'] = {}
+            if 'Unknown' not in functions['Unknown'].keys():
+                functions['Unknown']['Unknown'] = {}
+                functions['Unknown']['Unknown']['cycles'] = 0
+                functions['Unknown']['Unknown']['count'] = 0
+            functions['Unknown']['Unknown']['cycles'] += cycles
+
+            add_cycles_to_function.prev_func_name = None
+            return None
+
+        for func_key, module in functions_addr[key].iteritems():
+            for module_key, module_value in module.iteritems():
+                if (module_value['start'] <= addr) and (addr < module_value['end']):
+                    module_value['cycles'] += cycles
+
+                    # In case o <Unknown> we prefer to fallback on the direct search
+                    add_cycles_to_function.prev_func_name   = func_key
+                    add_cycles_to_function.prev_module_name = module_key
+                    add_cycles_to_function.prev_entry       = module_value
+                    return (func_key, module_key)
+
+    print "Warning: Function %s @ 0x%x not found" % (func_name, addr)
+
+    add_cycles_to_function.prev_func_name = None
+    return None
+
+# Static variables for the previous function
+add_cycles_to_function.prev_func_name = None
+add_cycles_to_function.prev_entry     = None
+
+def trace_read():
+    global trace_process
+    line = trace.readline()
+    trace_process += len(line)
+    return line
+
+#
+# Parse arguments
+#
+trace_name = None
+symbols_file = None
+
+opts,args = getopt.getopt(sys.argv[1:], "ht:vs:v", ["help","trace=","symbols="])
+if (opts is None) or (not opts):
+    usage()
+    sys.exit()
+
+for o,a in opts:
+    if o in ("-h","--help"):
+        usage()
+        sys.exit()
+    elif o in ("-t","--trace"):
+        trace_name = a
+    elif o in ("-s","--symbols"):
+        symbols_file = a
+    else:
+        assert False, "Unhandled option (%s)" % o
+
+#
+# We try first to see if we run the script from DS-5
+#
+try:
+    from arm_ds.debugger_v1 import Debugger
+    from arm_ds.debugger_v1 import DebugException
+
+    # Debugger object for accessing the debugger
+    debugger = Debugger()
+
+    # Initialisation commands
+    ec = debugger.getExecutionContext(0)
+    ec.getExecutionService().stop()
+    ec.getExecutionService().waitForStop()
+    # in case the execution context reference is out of date
+    ec = debugger.getExecutionContext(0)
+
+    #
+    # Get the module name and their memory range
+    #
+    info_file = ec.executeDSCommand("info file")
+    info_file_str = StringIO(info_file)
+
+    line = info_file_str.readline().strip('\n')
+    while line != '':
+        if ("Symbols from" in line):
+            # Get the module name from the line 'Symbols from "/home/...."'
+            module_name = line.split("\"")[1].split("/")[-1]
+            modules[module_name] = {}
+
+            # Look for the text section
+            line = info_file_str.readline().strip('\n')
+            while (line != '') and ("Symbols from" not in line):
+                if ("ER_RO" in line):
+                    modules[module_name]['start'] = get_address_from_string(line.split()[0])
+                    modules[module_name]['end']   = get_address_from_string(line.split()[2])
+                    line = info_file_str.readline().strip('\n')
+                    break;
+                if (".text" in line):
+                    modules[module_name]['start'] = get_address_from_string(line.split()[0])
+                    modules[module_name]['end']   = get_address_from_string(line.split()[2])
+                    line = info_file_str.readline().strip('\n')
+                    break;
+                line = info_file_str.readline().strip('\n')
+        line = info_file_str.readline().strip('\n')
+
+    #
+    # Get the function name and their memory range
+    #
+    info_func = ec.executeDSCommand("info func")
+    info_func_str = StringIO(info_func)
+
+    # Skip the first line 'Low-level symbols ...'
+    line = info_func_str.readline().strip('\n')
+    func_prev = None
+    while line != '':
+        # We ignore all the functions after 'Functions in'
+        if ("Functions in " in line):
+            line = info_func_str.readline().strip('\n')
+            while line != '':
+                line = info_func_str.readline().strip('\n')
+            line = info_func_str.readline().strip('\n')
+            continue
+
+        if ("Low-level symbols" in line):
+            # We need to fixup the last function of the module
+            if func_prev is not None:
+                func_prev['end'] = modules[module_name]['end']
+                func_prev = None
+
+            line = info_func_str.readline().strip('\n')
+            continue
+
+        func_name = line.split()[1]
+        func_start = get_address_from_string(line.split()[0])
+        module_name = get_module_from_addr(modules, func_start)
+
+        if func_name not in functions.keys():
+            functions[func_name] = {}
+        functions[func_name][module_name] = {}
+        functions[func_name][module_name]['start'] = func_start
+        functions[func_name][module_name]['cycles'] = 0
+        functions[func_name][module_name]['count'] = 0
+
+        # Set the end address of the previous function
+        if func_prev is not None:
+            func_prev['end'] = func_start
+        func_prev = functions[func_name][module_name]
+
+        line = info_func_str.readline().strip('\n')
+
+    # Fixup the last function
+    func_prev['end'] = modules[module_name]['end']
+
+    if symbols_file is not None:
+        pickle.dump((modules, functions), open(symbols_file, "w"))
+except:
+    if symbols_file is None:
+        print "Error: Symbols file is required when run out of ARM DS-5"
+        sys.exit()
+
+    (modules, functions) = pickle.load(open(symbols_file, "r"))
+
+#
+# Build optimized table for the <Unknown> functions
+#
+functions_addr = {}
+for func_key, module in functions.iteritems():
+    for module_key, module_value in module.iteritems():
+        key = module_value['start'] & ~0x0FFF
+        if key not in functions_addr.keys():
+            functions_addr[key] = {}
+        if func_key not in functions_addr[key].keys():
+            functions_addr[key][func_key] = {}
+        functions_addr[key][func_key][module_key] = module_value
+
+#
+# Process the trace file
+#
+if trace_name is None:
+    sys.exit()
+
+trace = open(trace_name, "r")
+trace_size = os.path.getsize(trace_name)
+trace_process = 0
+
+# Get the column names from the first line
+columns = trace_read().split()
+column_addr     = columns.index('Address')
+column_cycles   = columns.index('Cycles')
+column_function = columns.index('Function')
+
+line = trace_read()
+i = 0
+prev_callee = None
+while line:
+    try:
+        func_name = line.split('\t')[column_function].strip()
+        address   = get_address_from_string(line.split('\t')[column_addr])
+        cycles    = int(line.split('\t')[column_cycles])
+        callee = add_cycles_to_function(functions, func_name, address, cycles)
+        if (prev_callee != None) and (prev_callee != callee):
+            functions[prev_callee[0]][prev_callee[1]]['count'] += 1
+        prev_callee = callee
+    except ValueError:
+        pass
+    line = trace_read()
+    if ((i % 1000000) == 0) and (i != 0):
+        percent = (trace_process * 100.00) / trace_size
+        print "Processing file ... (%.2f %%)" % (percent)
+    i = i + 1
+
+# Fixup the last callee
+functions[prev_callee[0]][prev_callee[1]]['count'] += 1
+
+#
+# Process results
+#
+functions_cycles     = {}
+all_functions_cycles = {}
+total_cycles         = 0
+
+for func_key, module in functions.iteritems():
+    for module_key, module_value in module.iteritems():
+        key = "%s/%s" % (module_key, func_key)
+        functions_cycles[key] = (module_value['cycles'], module_value['count'])
+        total_cycles += module_value['cycles']
+
+        if func_key not in all_functions_cycles.keys():
+            all_functions_cycles[func_key] = (module_value['cycles'], module_value['count'])
+        else:
+            all_functions_cycles[func_key] = tuple(map(sum, zip(all_functions_cycles[func_key], (module_value['cycles'], module_value['count']))))
+
+sorted_functions_cycles     = sorted(functions_cycles.iteritems(), key=operator.itemgetter(1), reverse = True)
+sorted_all_functions_cycles = sorted(all_functions_cycles.items(), key=operator.itemgetter(1), reverse = True)
+
+print
+print "----"
+for (key,value) in sorted_functions_cycles[:20]:
+    if value[0] != 0:
+        print "%s (cycles: %d - %d%%, count: %d)" % (key, value[0], (value[0] * 100) / total_cycles, value[1])
+    else:
+        break;
+print "----"
+for (key,value) in sorted_all_functions_cycles[:20]:
+    if value[0] != 0:
+        print "%s (cycles: %d - %d%%, count: %d)" % (key, value[0], (value[0] * 100) / total_cycles, value[1])
+    else:
+        break;

ArmVirtPkg/ArmVirt.dsc.inc

@@ -165,13 +165,11 @@
   # Secure Boot dependencies
   #
 !if $(SECURE_BOOT_ENABLE) == TRUE
-  TpmMeasurementLib|SecurityPkg/Library/DxeTpmMeasurementLib/DxeTpmMeasurementLib.inf
   AuthVariableLib|SecurityPkg/Library/AuthVariableLib/AuthVariableLib.inf
 
   # re-use the UserPhysicalPresent() dummy implementation from the ovmf tree
   PlatformSecureLib|OvmfPkg/Library/PlatformSecureLib/PlatformSecureLib.inf
 !else
-  TpmMeasurementLib|MdeModulePkg/Library/TpmMeasurementLibNull/TpmMeasurementLibNull.inf
   AuthVariableLib|MdeModulePkg/Library/AuthVariableLibNull/AuthVariableLibNull.inf
 !endif
   VarCheckLib|MdeModulePkg/Library/VarCheckLib/VarCheckLib.inf
@@ -379,6 +377,10 @@
     <PcdsFixedAtBuild>
       gEfiShellPkgTokenSpaceGuid.PcdShellLibAutoInitialize|FALSE
   }
+  OvmfPkg/LinuxInitrdDynamicShellCommand/LinuxInitrdDynamicShellCommand.inf {
+    <PcdsFixedAtBuild>
+      gEfiShellPkgTokenSpaceGuid.PcdShellLibAutoInitialize|FALSE
+  }
   ShellPkg/Application/Shell/Shell.inf {
     <LibraryClasses>
       ShellCommandLib|ShellPkg/Library/UefiShellCommandLib/UefiShellCommandLib.inf

ArmVirtPkg/ArmVirtPkg.ci.yaml

@@ -0,0 +1,103 @@
+## @file
+# Core CI configuration for ArmVirtPkg
+#
+# ArmVirtPkg is part of Platform Ci for builds so this is only
+# used for code analysis.
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+    ## options defined .pytool/Plugin/CompilerPlugin
+    "CompilerPlugin": {
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+    "HostUnitTestCompilerPlugin": {
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/CharEncodingCheck
+    "CharEncodingCheck": {
+        "IgnoreFiles": []
+    },
+
+    ## options defined .pytool/Plugin/DependencyCheck
+    "DependencyCheck": {
+        "AcceptableDependencies": [
+            "MdePkg/MdePkg.dec",
+            "MdeModulePkg/MdeModulePkg.dec",
+            "ArmVirtPkg/ArmVirtPkg.dec",
+            "NetworkPkg/NetworkPkg.dec",
+            "ArmPkg/ArmPkg.dec",
+            "OvmfPkg/OvmfPkg.dec",
+            "EmbeddedPkg/EmbeddedPkg.dec",
+            "ArmPlatformPkg/ArmPlatformPkg.dec",
+            "SecurityPkg/SecurityPkg.dec",
+            "ShellPkg/ShellPkg.dec"  #Is this ok?
+        ],
+        # For host based unit tests
+        "AcceptableDependencies-HOST_APPLICATION":[
+            "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
+        ],
+        # For UEFI shell based apps
+        "AcceptableDependencies-UEFI_APPLICATION":[
+
+        ],
+        "IgnoreInf": []
+    },
+
+    ## options defined .pytool/Plugin/DscCompleteCheck
+    "DscCompleteCheck": {
+        "IgnoreInf": [""],
+        "DscPath": ""  # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
+    "HostUnitTestDscCompleteCheck": {
+        "IgnoreInf": [""],
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/GuidCheck
+    "GuidCheck": {
+        "IgnoreGuidName": [],
+        "IgnoreGuidValue": [],
+        "IgnoreFoldersAndFiles": [],
+        "IgnoreDuplicates": [],
+    },
+
+    ## options defined .pytool/Plugin/LibraryClassCheck
+    "LibraryClassCheck": {
+        "IgnoreHeaderFile": []
+    },
+
+    ## options defined .pytool/Plugin/SpellCheck
+    "SpellCheck": {
+        "AuditOnly": False,           # Fails right now with over 270 errors
+        "IgnoreFiles": [],           # use gitignore syntax to ignore errors in matching files
+        "ExtendWords": [
+            "setjump",
+            "plong",
+            "lparam",
+            "lpdword",
+            "lpthread",
+            "lresult",
+            "bootable",
+            "bsymbolic",
+            "endiannness",
+            "fvmain",
+            "multiboot",
+            "qemu's",
+            "ramdisk",
+            "ramfb",
+            "unbootable",
+            "virt's",
+            "werror",
+            "xenio"
+        ],           # words to extend to the dictionary for this package
+        "IgnoreStandardPaths": [],   # Standard Plugin defined paths that should be ignore
+        "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+    }
+}

ArmVirtPkg/ArmVirtPkg.dec

@@ -36,6 +36,12 @@
 [Protocols]
   gFdtClientProtocolGuid = { 0xE11FACA0, 0x4710, 0x4C8E, { 0xA7, 0xA2, 0x01, 0xBA, 0xA2, 0x59, 0x1B, 0x4C } }
 
+[PcdsFeatureFlag]
+  #
+  # Feature Flag PCD that defines whether TPM2 support is enabled
+  #
+  gArmVirtTokenSpaceGuid.PcdTpm2SupportEnabled|FALSE|BOOLEAN|0x00000004
+
 [PcdsFixedAtBuild, PcdsPatchableInModule]
   #
   # This is the physical address where the device tree is expected to be stored

ArmVirtPkg/ArmVirtQemu.dsc

@@ -29,6 +29,8 @@
   #
   DEFINE TTY_TERMINAL            = FALSE
   DEFINE SECURE_BOOT_ENABLE      = FALSE
+  DEFINE TPM2_ENABLE             = FALSE
+  DEFINE TPM2_CONFIG_ENABLE      = FALSE
 
   #
   # Network definition
@@ -56,6 +58,8 @@
   VirtioMmioDeviceLib|OvmfPkg/Library/VirtioMmioDeviceLib/VirtioMmioDeviceLib.inf
   QemuFwCfgLib|ArmVirtPkg/Library/QemuFwCfgLib/QemuFwCfgLib.inf
   QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/BaseQemuFwCfgS3LibNull.inf
+  QemuFwCfgSimpleParserLib|OvmfPkg/Library/QemuFwCfgSimpleParserLib/QemuFwCfgSimpleParserLib.inf
+  QemuLoadImageLib|OvmfPkg/Library/GenericQemuLoadImageLib/GenericQemuLoadImageLib.inf
 
   ArmPlatformLib|ArmPlatformPkg/Library/ArmPlatformLibNull/ArmPlatformLibNull.inf
 
@@ -74,12 +78,30 @@
   PciSegmentLib|MdePkg/Library/BasePciSegmentLibPci/BasePciSegmentLibPci.inf
   PciHostBridgeLib|ArmVirtPkg/Library/FdtPciHostBridgeLib/FdtPciHostBridgeLib.inf
 
+!if $(TPM2_ENABLE) == TRUE
+  Tpm2CommandLib|SecurityPkg/Library/Tpm2CommandLib/Tpm2CommandLib.inf
+  Tcg2PhysicalPresenceLib|OvmfPkg/Library/Tcg2PhysicalPresenceLibQemu/DxeTcg2PhysicalPresenceLib.inf
+  TpmMeasurementLib|SecurityPkg/Library/DxeTpmMeasurementLib/DxeTpmMeasurementLib.inf
+!else
+  TpmMeasurementLib|MdeModulePkg/Library/TpmMeasurementLibNull/TpmMeasurementLibNull.inf
+!endif
+
 [LibraryClasses.common.PEIM]
   ArmVirtMemInfoLib|ArmVirtPkg/Library/QemuVirtMemInfoLib/QemuVirtMemInfoPeiLib.inf
 
+!if $(TPM2_ENABLE) == TRUE
+  BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
+  ResetSystemLib|MdeModulePkg/Library/PeiResetSystemLib/PeiResetSystemLib.inf
+  Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibDTpm/Tpm2DeviceLibDTpm.inf
+!endif
+
 [LibraryClasses.common.DXE_DRIVER]
   ReportStatusCodeLib|MdeModulePkg/Library/DxeReportStatusCodeLib/DxeReportStatusCodeLib.inf
 
+!if $(TPM2_ENABLE) == TRUE
+  Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
+!endif
+
 [LibraryClasses.common.UEFI_DRIVER]
   UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
 
@@ -100,6 +122,8 @@
 
   gEfiMdeModulePkgTokenSpaceGuid.PcdTurnOffUsbLegacySupport|TRUE
 
+  gArmVirtTokenSpaceGuid.PcdTpm2SupportEnabled|$(TPM2_ENABLE)
+
 [PcdsFixedAtBuild.common]
 !if $(ARCH) == AARCH64
   gArmTokenSpaceGuid.PcdVFPEnabled|1
@@ -186,10 +210,6 @@
   # point only, for entry point versions >= 3.0.
   gEfiMdeModulePkgTokenSpaceGuid.PcdSmbiosEntryPointProvideMethod|0x2
 
-  # ACPI predates the AARCH64 architecture by 5 versions, so
-  # we only target OSes that support ACPI v5.0 or later
-  gEfiMdeModulePkgTokenSpaceGuid.PcdAcpiExposedTableVersions|0x20
-
 [PcdsDynamicDefault.common]
   gEfiMdePkgTokenSpaceGuid.PcdPlatformBootTimeOut|3
 
@@ -237,9 +257,29 @@
   gEfiMdeModulePkgTokenSpaceGuid.PcdSmbiosDocRev|0x0
   gUefiOvmfPkgTokenSpaceGuid.PcdQemuSmbiosValidated|FALSE
 
+  #
+  # IPv4 and IPv6 PXE Boot support.
+  #
+  gEfiNetworkPkgTokenSpaceGuid.PcdIPv4PXESupport|0x01
+  gEfiNetworkPkgTokenSpaceGuid.PcdIPv6PXESupport|0x01
+
+  #
+  # TPM2 support
+  #
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpmBaseAddress|0x0
+!if $(TPM2_ENABLE) == TRUE
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpmInstanceGuid|{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpm2HashMask|0
+!endif
+
 [PcdsDynamicHii]
   gArmVirtTokenSpaceGuid.PcdForceNoAcpi|L"ForceNoAcpi"|gArmVirtVariableGuid|0x0|FALSE|NV,BS
 
+!if $(TPM2_CONFIG_ENABLE) == TRUE
+  gEfiSecurityPkgTokenSpaceGuid.PcdTcgPhysicalPresenceInterfaceVer|L"TCG2_VERSION"|gTcg2ConfigFormSetGuid|0x0|"1.3"|NV,BS
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpm2AcpiTableRev|L"TCG2_VERSION"|gTcg2ConfigFormSetGuid|0x8|3|NV,BS
+!endif
+
 ################################################################################
 #
 # Components Section - list of all EDK II Modules needed by this Platform
@@ -261,6 +301,23 @@
 
   MdeModulePkg/Universal/Variable/Pei/VariablePei.inf
 
+!if $(TPM2_ENABLE) == TRUE
+  MdeModulePkg/Universal/ResetSystemPei/ResetSystemPei.inf {
+    <LibraryClasses>
+      ResetSystemLib|ArmVirtPkg/Library/ArmVirtPsciResetSystemPeiLib/ArmVirtPsciResetSystemPeiLib.inf
+  }
+  OvmfPkg/Tcg/Tcg2Config/Tcg2ConfigPei.inf
+  SecurityPkg/Tcg/Tcg2Pei/Tcg2Pei.inf {
+    <LibraryClasses>
+      HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterPei.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha1/HashInstanceLibSha1.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha256/HashInstanceLibSha256.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha384/HashInstanceLibSha384.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha512/HashInstanceLibSha512.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSm3/HashInstanceLibSm3.inf
+  }
+!endif
+
   MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf {
     <LibraryClasses>
       NULL|MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf
@@ -295,6 +352,9 @@
   MdeModulePkg/Universal/SecurityStubDxe/SecurityStubDxe.inf {
     <LibraryClasses>
       NULL|SecurityPkg/Library/DxeImageVerificationLib/DxeImageVerificationLib.inf
+!if $(TPM2_ENABLE) == TRUE
+      NULL|SecurityPkg/Library/DxeTpm2MeasureBootLib/DxeTpm2MeasureBootLib.inf
+!endif
   }
   SecurityPkg/VariableAuthenticated/SecureBootConfigDxe/SecureBootConfigDxe.inf
   OvmfPkg/EnrollDefaultKeys/EnrollDefaultKeys.inf
@@ -371,11 +431,18 @@
       NULL|MdeModulePkg/Library/BootManagerUiLib/BootManagerUiLib.inf
       NULL|MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib.inf
   }
+  OvmfPkg/QemuKernelLoaderFsDxe/QemuKernelLoaderFsDxe.inf
 
   #
   # Networking stack
   #
 !include NetworkPkg/NetworkComponents.dsc.inc
+
+  NetworkPkg/UefiPxeBcDxe/UefiPxeBcDxe.inf {
+    <LibraryClasses>
+      NULL|OvmfPkg/Library/PxeBcPcdProducerLib/PxeBcPcdProducerLib.inf
+  }
+
 !if $(NETWORK_TLS_ENABLE) == TRUE
   NetworkPkg/TlsAuthConfigDxe/TlsAuthConfigDxe.inf {
     <LibraryClasses>
@@ -389,6 +456,11 @@
   MdeModulePkg/Bus/Scsi/ScsiBusDxe/ScsiBusDxe.inf
   MdeModulePkg/Bus/Scsi/ScsiDiskDxe/ScsiDiskDxe.inf
 
+  #
+  # NVME Driver
+  #
+  MdeModulePkg/Bus/Pci/NvmExpressDxe/NvmExpressDxe.inf
+
   #
   # SMBIOS Support
   #
@@ -430,6 +502,26 @@
   MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
   MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
 
+  #
+  # TPM2 support
+  #
+!if $(TPM2_ENABLE) == TRUE
+  SecurityPkg/Tcg/Tcg2Dxe/Tcg2Dxe.inf {
+    <LibraryClasses>
+      HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
+      Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibRouter/Tpm2DeviceLibRouterDxe.inf
+      NULL|SecurityPkg/Library/Tpm2DeviceLibDTpm/Tpm2InstanceLibDTpm.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha1/HashInstanceLibSha1.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha256/HashInstanceLibSha256.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha384/HashInstanceLibSha384.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSha512/HashInstanceLibSha512.inf
+      NULL|SecurityPkg/Library/HashInstanceLibSm3/HashInstanceLibSm3.inf
+  }
+!if $(TPM2_CONFIG_ENABLE) == TRUE
+  SecurityPkg/Tcg/Tcg2Config/Tcg2ConfigDxe.inf
+!endif
+!endif
+
   #
   # ACPI Support
   #

ArmVirtPkg/ArmVirtQemu.fdf

@@ -113,6 +113,12 @@ READ_LOCK_STATUS   = TRUE
   INF MdeModulePkg/Universal/Variable/Pei/VariablePei.inf
   INF MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf
 
+!if $(TPM2_ENABLE) == TRUE
+  INF MdeModulePkg/Universal/ResetSystemPei/ResetSystemPei.inf
+  INF OvmfPkg/Tcg/Tcg2Config/Tcg2ConfigPei.inf
+  INF SecurityPkg/Tcg/Tcg2Pei/Tcg2Pei.inf
+!endif
+
   FILE FV_IMAGE = 9E21FD93-9C72-4c15-8C4B-E77F1DB2D792 {
     SECTION GUIDED EE4E5898-3914-4259-9D6E-DC7BD79403CF PROCESSING_REQUIRED = TRUE {
       SECTION FV_IMAGE = FVMAIN

ArmVirtPkg/ArmVirtQemuFvMain.fdf.inc

@@ -103,6 +103,7 @@ READ_LOCK_STATUS   = TRUE
   #
   INF ShellPkg/Application/Shell/Shell.inf
   INF ShellPkg/DynamicCommand/TftpDynamicCommand/TftpDynamicCommand.inf
+  INF OvmfPkg/LinuxInitrdDynamicShellCommand/LinuxInitrdDynamicShellCommand.inf
 
   #
   # Bds
@@ -113,6 +114,7 @@ READ_LOCK_STATUS   = TRUE
   INF MdeModulePkg/Universal/DriverHealthManagerDxe/DriverHealthManagerDxe.inf
   INF MdeModulePkg/Universal/BdsDxe/BdsDxe.inf
   INF MdeModulePkg/Application/UiApp/UiApp.inf
+  INF OvmfPkg/QemuKernelLoaderFsDxe/QemuKernelLoaderFsDxe.inf
 
   #
   # Networking stack
@@ -126,6 +128,11 @@ READ_LOCK_STATUS   = TRUE
   INF MdeModulePkg/Bus/Scsi/ScsiBusDxe/ScsiBusDxe.inf
   INF MdeModulePkg/Bus/Scsi/ScsiDiskDxe/ScsiDiskDxe.inf
 
+  #
+  # NVME Driver
+  #
+  INF MdeModulePkg/Bus/Pci/NvmExpressDxe/NvmExpressDxe.inf
+
   #
   # SMBIOS Support
   #
@@ -173,6 +180,16 @@ READ_LOCK_STATUS   = TRUE
   INF MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
   INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
 
+  #
+  # TPM2 support
+  #
+!if $(TPM2_ENABLE) == TRUE
+  INF SecurityPkg/Tcg/Tcg2Dxe/Tcg2Dxe.inf
+!if $(TPM2_CONFIG_ENABLE) == TRUE
+  INF SecurityPkg/Tcg/Tcg2Config/Tcg2ConfigDxe.inf
+!endif
+!endif
+
   #
   # TianoCore logo (splash screen)
   #

ArmVirtPkg/ArmVirtQemuKernel.dsc

@@ -56,6 +56,8 @@
   VirtioMmioDeviceLib|OvmfPkg/Library/VirtioMmioDeviceLib/VirtioMmioDeviceLib.inf
   QemuFwCfgLib|ArmVirtPkg/Library/QemuFwCfgLib/QemuFwCfgLib.inf
   QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/BaseQemuFwCfgS3LibNull.inf
+  QemuFwCfgSimpleParserLib|OvmfPkg/Library/QemuFwCfgSimpleParserLib/QemuFwCfgSimpleParserLib.inf
+  QemuLoadImageLib|OvmfPkg/Library/GenericQemuLoadImageLib/GenericQemuLoadImageLib.inf
 
   ArmVirtMemInfoLib|ArmVirtPkg/Library/QemuVirtMemInfoLib/QemuVirtMemInfoLib.inf
 
@@ -73,6 +75,7 @@
   PciPcdProducerLib|ArmVirtPkg/Library/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
   PciSegmentLib|MdePkg/Library/BasePciSegmentLibPci/BasePciSegmentLibPci.inf
   PciHostBridgeLib|ArmVirtPkg/Library/FdtPciHostBridgeLib/FdtPciHostBridgeLib.inf
+  TpmMeasurementLib|MdeModulePkg/Library/TpmMeasurementLibNull/TpmMeasurementLibNull.inf
 
 [LibraryClasses.common.DXE_DRIVER]
   ReportStatusCodeLib|MdeModulePkg/Library/DxeReportStatusCodeLib/DxeReportStatusCodeLib.inf
@@ -172,6 +175,12 @@
   gEfiMdePkgTokenSpaceGuid.PcdReportStatusCodePropertyMask|3
 
 [PcdsPatchableInModule.common]
+  # we need to provide a resolution for this PCD that supports PcdSet64()
+  # being called from ArmVirtPkg/Library/PlatformPeiLib/PlatformPeiLib.c,
+  # even though that call will be compiled out on this platform as it does
+  # not (and cannot) support the TPM2 driver stack
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpmBaseAddress|0x0
+
   #
   # This will be overridden in the code
   #
@@ -231,6 +240,12 @@
   gEfiMdeModulePkgTokenSpaceGuid.PcdSmbiosDocRev|0x0
   gUefiOvmfPkgTokenSpaceGuid.PcdQemuSmbiosValidated|FALSE
 
+  #
+  # IPv4 and IPv6 PXE Boot support.
+  #
+  gEfiNetworkPkgTokenSpaceGuid.PcdIPv4PXESupport|0x01
+  gEfiNetworkPkgTokenSpaceGuid.PcdIPv6PXESupport|0x01
+
 ################################################################################
 #
 # Components Section - list of all EDK II Modules needed by this Platform
@@ -355,11 +370,18 @@
       NULL|MdeModulePkg/Library/BootManagerUiLib/BootManagerUiLib.inf
       NULL|MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib.inf
   }
+  OvmfPkg/QemuKernelLoaderFsDxe/QemuKernelLoaderFsDxe.inf
 
   #
   # Networking stack
   #
 !include NetworkPkg/NetworkComponents.dsc.inc
+
+  NetworkPkg/UefiPxeBcDxe/UefiPxeBcDxe.inf {
+    <LibraryClasses>
+      NULL|OvmfPkg/Library/PxeBcPcdProducerLib/PxeBcPcdProducerLib.inf
+  }
+
 !if $(NETWORK_TLS_ENABLE) == TRUE
   NetworkPkg/TlsAuthConfigDxe/TlsAuthConfigDxe.inf {
     <LibraryClasses>
@@ -373,6 +395,11 @@
   MdeModulePkg/Bus/Scsi/ScsiBusDxe/ScsiBusDxe.inf
   MdeModulePkg/Bus/Scsi/ScsiDiskDxe/ScsiDiskDxe.inf
 
+  #
+  # NVME Driver
+  #
+  MdeModulePkg/Bus/Pci/NvmExpressDxe/NvmExpressDxe.inf
+
   #
   # SMBIOS Support
   #

ArmVirtPkg/ArmVirtXen.dsc

@@ -47,6 +47,7 @@
   BootLogoLib|MdeModulePkg/Library/BootLogoLib/BootLogoLib.inf
   PlatformBootManagerLib|ArmPkg/Library/PlatformBootManagerLib/PlatformBootManagerLib.inf
   CustomizedDisplayLib|MdeModulePkg/Library/CustomizedDisplayLib/CustomizedDisplayLib.inf
+  TpmMeasurementLib|MdeModulePkg/Library/TpmMeasurementLibNull/TpmMeasurementLibNull.inf
 
 [LibraryClasses.common.UEFI_DRIVER]
   UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
@@ -95,6 +96,12 @@
   gEfiMdeModulePkgTokenSpaceGuid.PcdEmuVariableNvModeEnable|TRUE
 
 [PcdsPatchableInModule.common]
+  # we need to provide a resolution for this PCD that supports PcdSet64()
+  # being called from ArmVirtPkg/Library/PlatformPeiLib/PlatformPeiLib.c,
+  # even though that call will be compiled out on this platform as it does
+  # not (and cannot) support the TPM2 driver stack
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpmBaseAddress|0x0
+
   #
   # This will be overridden in the code
   #

ArmVirtPkg/ArmVirtXen.fdf

@@ -182,6 +182,7 @@ READ_LOCK_STATUS   = TRUE
   #
   INF ShellPkg/Application/Shell/Shell.inf
   INF ShellPkg/DynamicCommand/TftpDynamicCommand/TftpDynamicCommand.inf
+  INF OvmfPkg/LinuxInitrdDynamicShellCommand/LinuxInitrdDynamicShellCommand.inf
 
   #
   # Bds

ArmVirtPkg/Include/Platform/Hidden.h

@@ -1,22 +1,22 @@
-/** @file
-
-  Copyright (c) 2018, Linaro Limited. All rights reserved.
-
-  SPDX-License-Identifier: BSD-2-Clause-Patent
-
-**/
-
-#ifndef __PLATFORM_HIDDEN_H
-#define __PLATFORM_HIDDEN_H
-
-//
-// Setting the GCC -fvisibility=hidden command line option is not quite the same
-// as setting the pragma below: the former only affects definitions, whereas the
-// pragma affects extern declarations as well. So if we want to ensure that no
-// GOT indirected symbol references are emitted, we need to use the pragma, or
-// GOT based cross object references could be emitted, e.g., in libraries, and
-// these cannot be relaxed to ordinary symbol references at link time.
-//
-#pragma GCC visibility push (hidden)
-
-#endif
+/** @file
+
+  Copyright (c) 2018, Linaro Limited. All rights reserved.
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef __PLATFORM_HIDDEN_H
+#define __PLATFORM_HIDDEN_H
+
+//
+// Setting the GCC -fvisibility=hidden command line option is not quite the same
+// as setting the pragma below: the former only affects definitions, whereas the
+// pragma affects extern declarations as well. So if we want to ensure that no
+// GOT indirected symbol references are emitted, we need to use the pragma, or
+// GOT based cross object references could be emitted, e.g., in libraries, and
+// these cannot be relaxed to ordinary symbol references at link time.
+//
+#pragma GCC visibility push (hidden)
+
+#endif

ArmVirtPkg/Library/ArmVirtGicArchLib/ArmVirtGicArchLib.c

@@ -110,7 +110,12 @@ ArmVirtGicArchLibConstructor (
     break;
 
   case 2:
-    ASSERT (RegSize == 32);
+    //
+    // When the GICv2 is emulated with virtualization=on, it adds a virtual
+    // set of control registers. This means the register property can be
+    // either 32 or 64 bytes in size.
+    //
+    ASSERT ((RegSize == 32) || (RegSize == 64));
 
     DistBase = SwapBytes64 (Reg[0]);
     CpuBase  = SwapBytes64 (Reg[2]);

ArmVirtPkg/Library/ArmVirtPsciResetSystemPeiLib/ArmVirtPsciResetSystemPeiLib.c

@@ -0,0 +1,232 @@
+/** @file
+  Reset System lib using PSCI hypervisor or secure monitor calls
+
+  Copyright (c) 2008 - 2009, Apple Inc. All rights reserved.<BR>
+  Copyright (c) 2013, ARM Ltd. All rights reserved.<BR>
+  Copyright (c) 2014-2020, Linaro Ltd. All rights reserved.<BR>
+  Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <PiPei.h>
+
+#include <libfdt.h>
+#include <Library/ArmHvcLib.h>
+#include <Library/ArmSmcLib.h>
+#include <Library/BaseLib.h>
+#include <Library/DebugLib.h>
+#include <Library/HobLib.h>
+#include <Library/ResetSystemLib.h>
+
+#include <IndustryStandard/ArmStdSmc.h>
+
+typedef enum {
+  PsciMethodUnknown,
+  PsciMethodSmc,
+  PsciMethodHvc,
+} PSCI_METHOD;
+
+STATIC
+PSCI_METHOD
+DiscoverPsciMethod (
+  VOID
+  )
+{
+  VOID                            *DeviceTreeBase;
+  INT32                           Node, Prev;
+  INT32                           Len;
+  CONST CHAR8                     *Compatible;
+  CONST CHAR8                     *CompatibleItem;
+  CONST VOID                      *Prop;
+
+  DeviceTreeBase = (VOID*)(UINTN)PcdGet64 (PcdDeviceTreeInitialBaseAddress);
+  ASSERT (fdt_check_header (DeviceTreeBase) == 0);
+
+  //
+  // Enumerate all FDT nodes looking for the PSCI node and capture the method
+  //
+  for (Prev = 0;; Prev = Node) {
+    Node = fdt_next_node (DeviceTreeBase, Prev, NULL);
+    if (Node < 0) {
+      break;
+    }
+
+    Compatible = fdt_getprop (DeviceTreeBase, Node, "compatible", &Len);
+    if (Compatible == NULL) {
+      continue;
+    }
+
+    //
+    // Iterate over the NULL-separated items in the compatible string
+    //
+    for (CompatibleItem = Compatible; CompatibleItem < Compatible + Len;
+      CompatibleItem += 1 + AsciiStrLen (CompatibleItem)) {
+
+      if (AsciiStrCmp (CompatibleItem, "arm,psci-0.2") != 0) {
+        continue;
+      }
+
+      Prop = fdt_getprop (DeviceTreeBase, Node, "method", NULL);
+      if (!Prop) {
+        DEBUG ((DEBUG_ERROR, "%a: Missing PSCI method property\n",
+          __FUNCTION__));
+        return PsciMethodUnknown;
+      }
+
+      if (AsciiStrnCmp (Prop, "hvc", 3) == 0) {
+        return PsciMethodHvc;
+      } else if (AsciiStrnCmp (Prop, "smc", 3) == 0) {
+        return PsciMethodSmc;
+      } else {
+        DEBUG ((DEBUG_ERROR, "%a: Unknown PSCI method \"%a\"\n", __FUNCTION__,
+          Prop));
+        return PsciMethodUnknown;
+      }
+    }
+  }
+  return PsciMethodUnknown;
+}
+
+STATIC
+VOID
+PerformPsciAction (
+  IN  UINTN     Arg0
+  )
+{
+  ARM_SMC_ARGS ArmSmcArgs;
+  ARM_HVC_ARGS ArmHvcArgs;
+
+  ArmSmcArgs.Arg0 = Arg0;
+  ArmHvcArgs.Arg0 = Arg0;
+
+  switch (DiscoverPsciMethod ()) {
+  case PsciMethodHvc:
+    ArmCallHvc (&ArmHvcArgs);
+    break;
+
+  case PsciMethodSmc:
+    ArmCallSmc (&ArmSmcArgs);
+    break;
+
+  default:
+    DEBUG ((DEBUG_ERROR, "%a: no PSCI method defined\n", __FUNCTION__));
+    ASSERT (FALSE);
+  }
+}
+
+/**
+  This function causes a system-wide reset (cold reset), in which
+  all circuitry within the system returns to its initial state. This type of reset
+  is asynchronous to system operation and operates without regard to
+  cycle boundaries.
+
+  If this function returns, it means that the system does not support cold reset.
+**/
+VOID
+EFIAPI
+ResetCold (
+  VOID
+  )
+{
+  // Send a PSCI 0.2 SYSTEM_RESET command
+  PerformPsciAction (ARM_SMC_ID_PSCI_SYSTEM_RESET);
+}
+
+/**
+  This function causes a system-wide initialization (warm reset), in which all processors
+  are set to their initial state. Pending cycles are not corrupted.
+
+  If this function returns, it means that the system does not support warm reset.
+**/
+VOID
+EFIAPI
+ResetWarm (
+  VOID
+  )
+{
+  // Map a warm reset into a cold reset
+  ResetCold ();
+}
+
+/**
+  This function causes the system to enter a power state equivalent
+  to the ACPI G2/S5 or G3 states.
+
+  If this function returns, it means that the system does not support shutdown reset.
+**/
+VOID
+EFIAPI
+ResetShutdown (
+  VOID
+  )
+{
+  // Send a PSCI 0.2 SYSTEM_OFF command
+  PerformPsciAction (ARM_SMC_ID_PSCI_SYSTEM_OFF);
+}
+
+/**
+  This function causes a systemwide reset. The exact type of the reset is
+  defined by the EFI_GUID that follows the Null-terminated Unicode string passed
+  into ResetData. If the platform does not recognize the EFI_GUID in ResetData
+  the platform must pick a supported reset type to perform.The platform may
+  optionally log the parameters from any non-normal reset that occurs.
+
+  @param[in]  DataSize   The size, in bytes, of ResetData.
+  @param[in]  ResetData  The data buffer starts with a Null-terminated string,
+                         followed by the EFI_GUID.
+**/
+VOID
+EFIAPI
+ResetPlatformSpecific (
+  IN UINTN   DataSize,
+  IN VOID    *ResetData
+  )
+{
+  // Map the platform specific reset as reboot
+  ResetCold ();
+}
+
+/**
+  The ResetSystem function resets the entire platform.
+
+  @param[in] ResetType      The type of reset to perform.
+  @param[in] ResetStatus    The status code for the reset.
+  @param[in] DataSize       The size, in bytes, of ResetData.
+  @param[in] ResetData      For a ResetType of EfiResetCold, EfiResetWarm, or EfiResetShutdown
+                            the data buffer starts with a Null-terminated string, optionally
+                            followed by additional binary data. The string is a description
+                            that the caller may use to further indicate the reason for the
+                            system reset.
+**/
+VOID
+EFIAPI
+ResetSystem (
+  IN EFI_RESET_TYPE               ResetType,
+  IN EFI_STATUS                   ResetStatus,
+  IN UINTN                        DataSize,
+  IN VOID                         *ResetData OPTIONAL
+  )
+{
+  switch (ResetType) {
+  case EfiResetWarm:
+    ResetWarm ();
+    break;
+
+  case EfiResetCold:
+    ResetCold ();
+    break;
+
+  case EfiResetShutdown:
+    ResetShutdown ();
+    return;
+
+  case EfiResetPlatformSpecific:
+    ResetPlatformSpecific (DataSize, ResetData);
+    return;
+
+  default:
+    return;
+  }
+}

ArmVirtPkg/Library/ArmVirtPsciResetSystemPeiLib/ArmVirtPsciResetSystemPeiLib.inf

@@ -0,0 +1,39 @@
+#/** @file
+# Reset System lib using PSCI hypervisor or secure monitor calls
+#
+# Copyright (c) 2008, Apple Inc. All rights reserved.<BR>
+# Copyright (c) 2014-2020, Linaro Ltd. All rights reserved.<BR>
+#
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+#
+#**/
+
+[Defines]
+  INF_VERSION                    = 1.27
+  BASE_NAME                      = ArmVirtPsciResetSystemPeiLib
+  FILE_GUID                      = 551cfb98-c185-41a3-86bf-8cdb7e2a530c
+  MODULE_TYPE                    = BASE
+  VERSION_STRING                 = 1.0
+  LIBRARY_CLASS                  = ResetSystemLib|PEIM
+
+[Sources]
+  ArmVirtPsciResetSystemPeiLib.c
+
+[Packages]
+  ArmPkg/ArmPkg.dec
+  ArmVirtPkg/ArmVirtPkg.dec
+  EmbeddedPkg/EmbeddedPkg.dec
+  MdeModulePkg/MdeModulePkg.dec
+  MdePkg/MdePkg.dec
+
+[LibraryClasses]
+  ArmSmcLib
+  ArmHvcLib
+  BaseLib
+  DebugLib
+  FdtLib
+  HobLib
+
+[Pcd]
+  gArmVirtTokenSpaceGuid.PcdDeviceTreeInitialBaseAddress

ArmVirtPkg/Library/PlatformBootManagerLib/PlatformBm.c

@@ -25,6 +25,7 @@
 #include <Protocol/PciRootBridgeIo.h>
 #include <Protocol/VirtioDevice.h>
 #include <Guid/EventGroup.h>
+#include <Guid/GlobalVariable.h>
 #include <Guid/RootBridgesConnectedEventGroup.h>
 #include <Guid/SerialPortLibVendor.h>
 
@@ -686,7 +687,9 @@ PlatformBootManagerBeforeConsole (
   VOID
   )
 {
+  UINT16        FrontPageTimeout;
   RETURN_STATUS PcdStatus;
+  EFI_STATUS    Status;
 
   //
   // Signal EndOfDxe PI Event
@@ -744,9 +747,29 @@ PlatformBootManagerBeforeConsole (
   //
   // Set the front page timeout from the QEMU configuration.
   //
-  PcdStatus = PcdSet16S (PcdPlatformBootTimeOut,
-                GetFrontPageTimeoutFromQemu ());
+  FrontPageTimeout = GetFrontPageTimeoutFromQemu ();
+  PcdStatus = PcdSet16S (PcdPlatformBootTimeOut, FrontPageTimeout);
   ASSERT_RETURN_ERROR (PcdStatus);
+  //
+  // Reflect the PCD in the standard Timeout variable.
+  //
+  Status = gRT->SetVariable (
+                  EFI_TIME_OUT_VARIABLE_NAME,
+                  &gEfiGlobalVariableGuid,
+                  (EFI_VARIABLE_NON_VOLATILE |
+                   EFI_VARIABLE_BOOTSERVICE_ACCESS |
+                   EFI_VARIABLE_RUNTIME_ACCESS),
+                  sizeof FrontPageTimeout,
+                  &FrontPageTimeout
+                  );
+  DEBUG ((
+    EFI_ERROR (Status) ? DEBUG_ERROR : DEBUG_VERBOSE,
+    "%a: SetVariable(%s, %u): %r\n",
+    __FUNCTION__,
+    EFI_TIME_OUT_VARIABLE_NAME,
+    FrontPageTimeout,
+    Status
+    ));
 
   //
   // Register platform-specific boot options and keyboard shortcuts.

ArmVirtPkg/Library/PlatformBootManagerLib/PlatformBootManagerLib.inf

@@ -44,9 +44,8 @@
   MemoryAllocationLib
   PcdLib
   PlatformBmPrintScLib
-  PrintLib
   QemuBootOrderLib
-  QemuFwCfgLib
+  QemuLoadImageLib
   ReportStatusCodeLib
   UefiBootManagerLib
   UefiBootServicesTableLib
@@ -64,18 +63,13 @@
   gEfiMdePkgTokenSpaceGuid.PcdPlatformBootTimeOut
 
 [Guids]
-  gEfiFileInfoGuid
-  gEfiFileSystemInfoGuid
-  gEfiFileSystemVolumeLabelInfoIdGuid
   gEfiEndOfDxeEventGroupGuid
+  gEfiGlobalVariableGuid
   gRootBridgesConnectedEventGroupGuid
   gUefiShellFileGuid
 
 [Protocols]
-  gEfiDevicePathProtocolGuid
   gEfiFirmwareVolume2ProtocolGuid
   gEfiGraphicsOutputProtocolGuid
-  gEfiLoadedImageProtocolGuid
   gEfiPciRootBridgeIoProtocolGuid
-  gEfiSimpleFileSystemProtocolGuid
   gVirtioDeviceProtocolGuid

ArmVirtPkg/Library/PlatformPeiLib/PlatformPeiLib.c

@@ -1,7 +1,7 @@
 /** @file
 *
 *  Copyright (c) 2011-2014, ARM Limited. All rights reserved.
-*  Copyright (c) 2014, Linaro Limited. All rights reserved.
+*  Copyright (c) 2014-2020, Linaro Limited. All rights reserved.
 *
 *  SPDX-License-Identifier: BSD-2-Clause-Patent
 *
@@ -13,11 +13,24 @@
 #include <Library/DebugLib.h>
 #include <Library/HobLib.h>
 #include <Library/PcdLib.h>
+#include <Library/PeiServicesLib.h>
 #include <libfdt.h>
 
 #include <Guid/EarlyPL011BaseAddress.h>
 #include <Guid/FdtHob.h>
 
+STATIC CONST EFI_PEI_PPI_DESCRIPTOR mTpm2DiscoveredPpi = {
+  EFI_PEI_PPI_DESCRIPTOR_PPI | EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST,
+  &gOvmfTpmDiscoveredPpiGuid,
+  NULL
+};
+
+STATIC CONST EFI_PEI_PPI_DESCRIPTOR mTpm2InitializationDonePpi = {
+  EFI_PEI_PPI_DESCRIPTOR_PPI | EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST,
+  &gPeiTpmInitializationDonePpiGuid,
+  NULL
+};
+
 EFI_STATUS
 EFIAPI
 PlatformPeim (
@@ -31,14 +44,18 @@ PlatformPeim (
   UINT64             *FdtHobData;
   UINT64             *UartHobData;
   INT32              Node, Prev;
+  INT32              Parent, Depth;
   CONST CHAR8        *Compatible;
   CONST CHAR8        *CompItem;
   CONST CHAR8        *NodeStatus;
   INT32              Len;
+  INT32              RangesLen;
   INT32              StatusLen;
   CONST UINT64       *RegProp;
+  CONST UINT32       *RangesProp;
   UINT64             UartBase;
-
+  UINT64             TpmBase;
+  EFI_STATUS         Status;
 
   Base = (VOID*)(UINTN)PcdGet64 (PcdDeviceTreeInitialBaseAddress);
   ASSERT (Base != NULL);
@@ -58,18 +75,23 @@ PlatformPeim (
   ASSERT (UartHobData != NULL);
   *UartHobData = 0;
 
+  TpmBase = 0;
+
   //
-  // Look for a UART node
+  // Set Parent to suppress incorrect compiler/analyzer warnings.
   //
-  for (Prev = 0;; Prev = Node) {
-    Node = fdt_next_node (Base, Prev, NULL);
+  Parent = 0;
+
+  for (Prev = Depth = 0;; Prev = Node) {
+    Node = fdt_next_node (Base, Prev, &Depth);
     if (Node < 0) {
       break;
     }
 
-    //
-    // Check for UART node
-    //
+    if (Depth == 1) {
+      Parent = Node;
+    }
+
     Compatible = fdt_getprop (Base, Node, "compatible", &Len);
 
     //
@@ -93,10 +115,74 @@ PlatformPeim (
 
         *UartHobData = UartBase;
         break;
+      } else if (FeaturePcdGet (PcdTpm2SupportEnabled) &&
+                 AsciiStrCmp (CompItem, "tcg,tpm-tis-mmio") == 0) {
+
+        RegProp = fdt_getprop (Base, Node, "reg", &Len);
+        ASSERT (Len == 8 || Len == 16);
+        if (Len == 8) {
+          TpmBase = fdt32_to_cpu (RegProp[0]);
+        } else if (Len == 16) {
+          TpmBase = fdt64_to_cpu (ReadUnaligned64 ((UINT64 *)RegProp));
+        }
+
+        if (Depth > 1) {
+          //
+          // QEMU/mach-virt may put the TPM on the platform bus, in which case
+          // we have to take its 'ranges' property into account to translate the
+          // MMIO address. This consists of a <child base, parent base, size>
+          // tuple, where the child base and the size use the same number of
+          // cells as the 'reg' property above, and the parent base uses 2 cells
+          //
+          RangesProp = fdt_getprop (Base, Parent, "ranges", &RangesLen);
+          ASSERT (RangesProp != NULL);
+
+          //
+          // a plain 'ranges' attribute without a value implies a 1:1 mapping
+          //
+          if (RangesLen != 0) {
+            //
+            // assume a single translated range with 2 cells for the parent base
+            //
+            if (RangesLen != Len + 2 * sizeof (UINT32)) {
+              DEBUG ((DEBUG_WARN,
+                "%a: 'ranges' property has unexpected size %d\n",
+                __FUNCTION__, RangesLen));
+              break;
+            }
+
+            if (Len == 8) {
+              TpmBase -= fdt32_to_cpu (RangesProp[0]);
+            } else {
+              TpmBase -= fdt64_to_cpu (ReadUnaligned64 ((UINT64 *)RangesProp));
+            }
+
+            //
+            // advance RangesProp to the parent bus address
+            //
+            RangesProp = (UINT32 *)((UINT8 *)RangesProp + Len / 2);
+            TpmBase += fdt64_to_cpu (ReadUnaligned64 ((UINT64 *)RangesProp));
+          }
+        }
+        break;
       }
     }
   }
 
+  if (FeaturePcdGet (PcdTpm2SupportEnabled)) {
+    if (TpmBase != 0) {
+      DEBUG ((DEBUG_INFO, "%a: TPM @ 0x%lx\n", __FUNCTION__, TpmBase));
+
+      Status = (EFI_STATUS)PcdSet64S (PcdTpmBaseAddress, TpmBase);
+      ASSERT_EFI_ERROR (Status);
+
+      Status = PeiServicesInstallPpi (&mTpm2DiscoveredPpi);
+    } else {
+      Status = PeiServicesInstallPpi (&mTpm2InitializationDonePpi);
+    }
+    ASSERT_EFI_ERROR (Status);
+  }
+
   BuildFvHob (PcdGet64 (PcdFvBaseAddress), PcdGet32 (PcdFvSize));
 
   return EFI_SUCCESS;

ArmVirtPkg/Library/PlatformPeiLib/PlatformPeiLib.inf

@@ -1,7 +1,7 @@
 #/** @file
 #
 #  Copyright (c) 2011-2015, ARM Limited. All rights reserved.
-#  Copyright (c) 2014, Linaro Limited. All rights reserved.
+#  Copyright (c) 2014-2020, Linaro Limited. All rights reserved.
 #
 #  SPDX-License-Identifier: BSD-2-Clause-Patent
 #
@@ -11,7 +11,7 @@
   INF_VERSION                    = 0x00010005
   BASE_NAME                      = PlatformPeiLib
   FILE_GUID                      = 59C11815-F8DA-4F49-B4FB-EC1E41ED1F06
-  MODULE_TYPE                    = SEC
+  MODULE_TYPE                    = BASE
   VERSION_STRING                 = 1.0
   LIBRARY_CLASS                  = PlatformPeiLib
 
@@ -21,14 +21,21 @@
 [Packages]
   ArmPkg/ArmPkg.dec
   ArmVirtPkg/ArmVirtPkg.dec
-  MdePkg/MdePkg.dec
-  MdeModulePkg/MdeModulePkg.dec
   EmbeddedPkg/EmbeddedPkg.dec
+  MdeModulePkg/MdeModulePkg.dec
+  MdePkg/MdePkg.dec
+  OvmfPkg/OvmfPkg.dec
+  SecurityPkg/SecurityPkg.dec
+
+[FeaturePcd]
+  gArmVirtTokenSpaceGuid.PcdTpm2SupportEnabled
 
 [LibraryClasses]
   DebugLib
   HobLib
   FdtLib
+  PcdLib
+  PeiServicesLib
 
 [FixedPcd]
   gArmTokenSpaceGuid.PcdFvSize
@@ -37,6 +44,11 @@
 [Pcd]
   gArmTokenSpaceGuid.PcdFvBaseAddress
   gArmVirtTokenSpaceGuid.PcdDeviceTreeInitialBaseAddress
+  gEfiSecurityPkgTokenSpaceGuid.PcdTpmBaseAddress         ## SOMETIMES_PRODUCES
+
+[Ppis]
+  gOvmfTpmDiscoveredPpiGuid                               ## SOMETIMES_PRODUCES
+  gPeiTpmInitializationDonePpiGuid                        ## SOMETIMES_PRODUCES
 
 [Guids]
   gEarlyPL011BaseAddressGuid

ArmVirtPkg/Library/QemuFwCfgLib/QemuFwCfgLib.inf

@@ -15,7 +15,7 @@
   FILE_GUID                      = B271F41F-B841-48A9-BA8D-545B4BC2E2BF
   MODULE_TYPE                    = BASE
   VERSION_STRING                 = 1.0
-  LIBRARY_CLASS                  = QemuFwCfgLib|DXE_DRIVER
+  LIBRARY_CLASS                  = QemuFwCfgLib|DXE_DRIVER UEFI_DRIVER
 
   CONSTRUCTOR                    = QemuFwCfgInitialize
 

ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml

@@ -0,0 +1,89 @@
+## @file
+# Azure Pipeline build file for building a platform.
+#
+# Platform: ArmVirtQemu
+# OS: Ubuntu
+# Toolchain: GCC5
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+  - master
+pr:
+  - master
+
+jobs:
+  - job: Platform_CI
+    variables:
+      package: 'ArmVirtPkg'
+      vm_image: 'ubuntu-latest'
+      should_run: true
+      run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
+
+    #Use matrix to speed up the build process
+    strategy:
+        matrix:
+          QEMU_AARCH64_DEBUG:
+            Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+            Build.Arch: "AARCH64"
+            Build.Flags: ""
+            Build.Target: "DEBUG"
+            Run.Flags: $(run_flags)
+            Run: $(should_run)
+          QEMU_AARCH64_RELEASE:
+            Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+            Build.Arch: "AARCH64"
+            Build.Flags: ""
+            Build.Target: "RELEASE"
+            Run.Flags: $(run_flags)
+            Run: $(should_run)
+          QEMU_AARCH64_NOOPT:
+            Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+            Build.Arch: "AARCH64"
+            Build.Flags: ""
+            Build.Target: "NOOPT"
+            Run.Flags: $(run_flags)
+            Run: $(should_run)
+          QEMU_ARM_DEBUG:
+            Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+            Build.Arch: "ARM"
+            Build.Flags: ""
+            Build.Target: "DEBUG"
+            Run.Flags: $(run_flags)
+            Run: $(should_run)
+          QEMU_ARM_RELEASE:
+            Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+            Build.Arch: "ARM"
+            Build.Flags: ""
+            Build.Target: "RELEASE"
+            Run.Flags: $(run_flags)
+            Run: $(should_run)
+          QEMU_ARM_NOOPT:
+            Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+            Build.Arch: "ARM"
+            Build.Flags: ""
+            Build.Target: "NOOPT"
+            Run.Flags: $(run_flags)
+            Run: $(should_run)
+
+    workspace:
+      clean: all
+
+    pool:
+      vmImage: $(vm_image)
+
+    steps:
+    - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
+      parameters:
+        tool_chain_tag: GCC5
+        build_pkg: $(package)
+        build_target: $(Build.Target)
+        build_arch: $(Build.Arch)
+        build_file: $(Build.File)
+        build_flags: $(Build.Flags)
+        run_flags: $(Run.Flags)
+        extra_install_step:
+        - bash: sudo apt-get install qemu
+          displayName: Install qemu
+          condition: and(gt(variables.pkg_count, 0), succeeded())

ArmVirtPkg/PlatformCI/PlatformBuild.py

@@ -0,0 +1,276 @@
+# @file
+# Script to Build ArmVirtPkg UEFI firmware
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import io
+
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment.uefi_build import UefiBuilder
+from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import RunCmd
+from edk2toollib.utility_functions import GetHostInfo
+
+# ####################################################################################### #
+#                                Common Configuration                                     #
+# ####################################################################################### #
+
+
+class CommonPlatform():
+    ''' Common settings for this platform.  Define static data here and use
+        for the different parts of stuart
+    '''
+    PackagesSupported = ("ArmVirtPkg",)
+    ArchSupported = ("AARCH64", "ARM")
+    TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
+    Scopes = ('armvirt', 'edk2-build')
+    WorkspaceRoot = os.path.realpath(os.path.join(
+        os.path.dirname(os.path.abspath(__file__)), "..", ".."))
+
+    # ####################################################################################### #
+    #                         Configuration for Update & Setup                                #
+    # ####################################################################################### #
+
+
+class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+    def GetPackagesSupported(self):
+        ''' return iterable of edk2 packages supported by this build.
+        These should be edk2 workspace relative paths '''
+        return CommonPlatform.PackagesSupported
+
+    def GetArchitecturesSupported(self):
+        ''' return iterable of edk2 architectures supported by this build '''
+        return CommonPlatform.ArchSupported
+
+    def GetTargetsSupported(self):
+        ''' return iterable of edk2 target tags supported by this build '''
+        return CommonPlatform.TargetsSupported
+
+    def GetRequiredSubmodules(self):
+        ''' return iterable containing RequiredSubmodule objects.
+        If no RequiredSubmodules return an empty iterable
+        '''
+        rs = []
+
+        # intentionally declare this one with recursive false to avoid overhead
+        rs.append(RequiredSubmodule(
+            "CryptoPkg/Library/OpensslLib/openssl", False))
+
+        # To avoid maintenance of this file for every new submodule
+        # lets just parse the .gitmodules and add each if not already in list.
+        # The GetRequiredSubmodules is designed to allow a build to optimize
+        # the desired submodules but it isn't necessary for this repository.
+        result = io.StringIO()
+        ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
+        # Cmd output is expected to look like:
+        # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
+        # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
+        if ret == 0:
+            for line in result.getvalue().splitlines():
+                _, _, path = line.partition(" ")
+                if path is not None:
+                    if path not in [x.path for x in rs]:
+                        rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
+        return rs
+
+    def SetArchitectures(self, list_of_requested_architectures):
+        ''' Confirm the requests architecture list is valid and configure SettingsManager
+        to run only the requested architectures.
+
+        Raise Exception if a list_of_requested_architectures is not supported
+        '''
+        unsupported = set(list_of_requested_architectures) - \
+            set(self.GetArchitecturesSupported())
+        if(len(unsupported) > 0):
+            errorString = (
+                "Unsupported Architecture Requested: " + " ".join(unsupported))
+            logging.critical(errorString)
+            raise Exception(errorString)
+        self.ActualArchitectures = list_of_requested_architectures
+
+    def GetWorkspaceRoot(self):
+        ''' get WorkspacePath '''
+        return CommonPlatform.WorkspaceRoot
+
+    def GetActiveScopes(self):
+        ''' return tuple containing scopes that should be active for this process '''
+
+        scopes = CommonPlatform.Scopes
+        ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+
+        if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
+            if "AARCH64" in self.ActualArchitectures:
+                scopes += ("gcc_aarch64_linux",)
+            if "ARM" in self.ActualArchitectures:
+                scopes += ("gcc_arm_linux",)
+        return scopes
+
+    def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+        ''' Filter other cases that this package should be built
+        based on changed files. This should cover things that can't
+        be detected as dependencies. '''
+        build_these_packages = []
+        possible_packages = potentialPackagesList.copy()
+        for f in changedFilesList:
+            # BaseTools files that might change the build
+            if "BaseTools" in f:
+                if os.path.splitext(f) not in [".txt", ".md"]:
+                    build_these_packages = possible_packages
+                    break
+
+            # if the azure pipeline platform template file changed
+            if "platform-build-run-steps.yml" in f:
+                build_these_packages = possible_packages
+                break
+
+
+        return build_these_packages
+
+    def GetPlatformDscAndConfig(self) -> tuple:
+        ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
+        any of the changes will be built in the dsc.
+
+        The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
+        '''
+        return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {})
+
+
+    # ####################################################################################### #
+    #                         Actual Configuration for Platform Build                         #
+    # ####################################################################################### #
+
+
+class PlatformBuilder(UefiBuilder, BuildSettingsManager):
+    def __init__(self):
+        UefiBuilder.__init__(self)
+
+    def AddCommandLineOptions(self, parserObj):
+        ''' Add command line options to the argparser '''
+        parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64",
+                               help="Optional - Architecture to build.  Default = AARCH64")
+
+    def RetrieveCommandLineOptions(self, args):
+        '''  Retrieve command line options from the argparser '''
+
+        shell_environment.GetBuildVars().SetValue(
+            "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
+
+        shell_environment.GetBuildVars().SetValue(
+            "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine")
+
+    def GetWorkspaceRoot(self):
+        ''' get WorkspacePath '''
+        return CommonPlatform.WorkspaceRoot
+
+    def GetPackagesPath(self):
+        ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
+        return ()
+
+    def GetActiveScopes(self):
+        ''' return tuple containing scopes that should be active for this process '''
+        scopes = CommonPlatform.Scopes
+        ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+        Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "")
+
+        if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
+            if "AARCH64" == Arch:
+                scopes += ("gcc_aarch64_linux",)
+            elif "ARM" == Arch:
+                scopes += ("gcc_arm_linux",)
+        return scopes
+
+    def GetName(self):
+        ''' Get the name of the repo, platform, or product being build '''
+        ''' Used for naming the log file, among others '''
+        # check the startup nsh flag and if set then rename the log file.
+        # this helps in CI so we don't overwrite the build log since running
+        # uses the stuart_build command.
+        if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
+            return "ArmVirtPkg_With_Run"
+        return "ArmVirtPkg"
+
+    def GetLoggingLevel(self, loggerType):
+        ''' Get the logging level for a given type
+        base == lowest logging level supported
+        con  == Screen logging
+        txt  == plain text file logging
+        md   == markdown file logging
+        '''
+        return logging.DEBUG
+
+    def SetPlatformEnv(self):
+        logging.debug("PlatformBuilder SetPlatformEnv")
+        self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded")
+        self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
+        self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
+        return 0
+
+    def PlatformPreBuild(self):
+        return 0
+
+    def PlatformPostBuild(self):
+        return 0
+
+    def FlashRomImage(self):
+        VirtualDrive = os.path.join(self.env.GetValue(
+            "BUILD_OUTPUT_BASE"), "VirtualDrive")
+        os.makedirs(VirtualDrive, exist_ok=True)
+        OutputPath_FV = os.path.join(
+            self.env.GetValue("BUILD_OUTPUT_BASE"), "FV")
+        Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd")
+
+        # pad fd to 64mb
+        with open(Built_FV, "ab") as fvfile:
+            fvfile.seek(0, os.SEEK_END)
+            additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell())
+            fvfile.write(additional)
+
+        # QEMU must be on that path
+
+        # Unique Command and Args parameters per ARCH
+        if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"):
+            cmd = "qemu-system-aarch64"
+            args = "-M virt"
+            args += " -cpu cortex-a57"                                          # emulate cpu
+        elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"):
+            cmd = "qemu-system-arm"
+            args = "-M virt"
+            args += " -cpu cortex-a15"                                          # emulate cpu
+        else:
+            raise NotImplementedError()
+
+        # Common Args
+        args += " -pflash " + Built_FV                                     # path to fw
+        args += " -m 1024"                                                  # 1gb memory
+        # turn off network
+        args += " -net none"
+        # Serial messages out
+        args += " -serial stdio"
+        # Mount disk with startup.nsh
+        args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk"
+
+        # Conditional Args
+        if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
+            args += " -display none"  # no graphics
+
+        if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
+            f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
+            f.write("BOOT SUCCESS !!! \n")
+            # add commands here
+            f.write("reset -s\n")
+            f.close()
+
+        ret = RunCmd(cmd, args)
+
+        if ret == 0xc0000005:
+            # for some reason getting a c0000005 on successful return
+            return 0
+
+        return ret

ArmVirtPkg/PlatformCI/ReadMe.md

@@ -0,0 +1,125 @@
+# ArmVirtPkg - Platform CI
+
+This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how
+to use the same Pytools based build infrastructure locally.
+
+## Supported Configuration Details
+
+This solution for building and running ArmVirtPkg has only been validated with Ubuntu
+18.04 and the GCC5 toolchain. Two different firmware builds are supported and are
+described below.
+
+| Configuration name      | Architecture       | DSC File         |Additional Flags |
+| :----------             | :-----             | :-----           | :----           |
+| AARCH64                 | AARCH64            | ArmVirtQemu.dsc  | None            |
+| ARM                     | ARM                | ArmVirtQemu.dsc  | None            |
+
+## EDK2 Developer environment
+
+- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
+- [GIT - Download & Install](https://git-scm.com/download/)
+- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/)
+- [Edk2 Source](https://github.com/tianocore/edk2)
+- Additional packages found necessary for Ubuntu 18.04
+  - apt-get install gcc g++ make uuid-dev
+
+Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
+the required cross-compiler toolchains are **not** required, this is handled by the
+Pytools build system.
+
+## Building with Pytools for ArmVirtPkg
+
+1. [Optional] Create a Python Virtual Environment - generally once per workspace
+
+    ``` bash
+    python -m venv <name of virtual environment>
+    ```
+
+2. [Optional] Activate Virtual Environment - each time new shell opened
+    - Windows
+
+      ``` bash
+      <name of virtual environment>/Scripts/activate.bat
+      ```
+
+    - Linux
+
+      ```bash
+      source <name of virtual environment>/bin/activate
+      ```
+
+3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
+
+    ``` bash
+    pip install --upgrade -r pip-requirements.txt
+    ```
+
+4. Initialize & Update Submodules - only when submodules updated
+
+    ``` bash
+    stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+    ```
+
+5. Initialize & Update Dependencies - only as needed when ext_deps change
+
+    ``` bash
+    stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+    ```
+
+6. Compile the basetools if necessary - only when basetools C source files change
+
+    ``` bash
+    python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
+    ```
+
+7. Compile Firmware
+
+    ``` bash
+    stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+    ```
+
+    - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional
+    options like `--clean`
+
+8. Running Emulator
+    - You can add `--FlashRom` to the end of your build command and the emulator will run after the
+    build is complete.
+    - or use the `--FlashOnly` feature to just run the emulator.
+
+      ``` bash
+      stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
+      ```
+
+### Notes
+
+1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical.
+   This is because there are extra scopes and tools that will be resolved during the update step that
+   need to match your build step.
+2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
+   environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
+3. QEMU must be on your path.  On Windows this is a manual process and not part of the QEMU installer.
+
+**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
+
+### Custom Build Options
+
+**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
+used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute
+the contents of *startup.nsh*.
+
+**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise
+an error occurs. Locally you don't need to set this.
+
+### Passing Build Defines
+
+To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
+command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
+For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build
+command-line would be:
+
+`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE`
+
+## References
+
+- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
+- More on [python virtual environments](https://docs.python.org/3/library/venv.html)

ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml

@@ -0,0 +1,21 @@
+## @file
+# Download iasl executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Add the folder with the tool to the path
+#
+# This is only downloaded for scope armvirt thus
+# should have no impact on the asl compiler used by any
+# other platform build
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+  "id": "iasl-armvirt-1",
+  "scope": "armvirt",
+  "type": "nuget",
+  "name": "iasl",
+  "source": "https://api.nuget.org/v3/index.json",
+  "version": "20190215.0.0",
+  "flags": ["set_path", "host_specific"],
+}

BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml

@@ -0,0 +1,22 @@
+## @file
+# Download GCC RISCV64 compiler from RISC-V Organization release site
+# Set shell variable GCC5_RISCV64_INSTALL to this folder
+#
+# This is only downloaded when a build activates scope gcc_riscv64_unknown
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+  "scope": "gcc_riscv64_unknown",
+  "type": "web",
+  "name": "gcc_riscv64_unknown",
+  "source": "https://raw.githubusercontent.com/riscv/riscv-uefi-edk2-docs/master/gcc-riscv-edk2-ci-toolchain/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu.tar.xz",
+  "version": "9.2.0",
+  "compression_type": "tar",
+  "sha256": "28373643b69f0ce008273c3dc63f172aa1121952f1b9ae94d7485ac94af7f344",
+  "internal_path": "/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu",
+  "flags": ["set_shell_var", ],
+  "var_name": "GCC5_RISCV64_INSTALL"
+}

BaseTools/Conf/tools_def.template

@@ -1,9 +1,9 @@
 #
 #  Copyright (c) 2006 - 2018, Intel Corporation. All rights reserved.<BR>
 #  Portions copyright (c) 2008 - 2009, Apple Inc. All rights reserved.<BR>
-#  Portions copyright (c) 2011 - 2014, ARM Ltd. All rights reserved.<BR>
+#  Portions copyright (c) 2011 - 2019, ARM Ltd. All rights reserved.<BR>
 #  Copyright (c) 2015, Hewlett-Packard Development Company, L.P.<BR>
-#  (C) Copyright 2016 Hewlett Packard Enterprise Development LP<BR>
+#  (C) Copyright 2020, Hewlett Packard Enterprise Development LP<BR>
 #  Copyright (c) Microsoft Corporation
 #
 #  SPDX-License-Identifier: BSD-2-Clause-Patent
@@ -267,11 +267,12 @@ DEFINE DTC_BIN                 = ENV(DTC_PREFIX)dtc
 #                               Intel(r) ACPI Compiler from
 #                               https://acpica.org/downloads
 #   GCC5        -Linux,Windows-  Requires:
-#                             GCC 5 with LTO support, targeting x86_64-linux-gnu, aarch64-linux-gnu, or arm-linux-gnueabi
+#                             GCC 5 with LTO support, targeting x86_64-linux-gnu, aarch64-linux-gnu, arm-linux-gnueabi or riscv64-linux-gnu
 #                        Optional:
 #                             Required to build platforms or ACPI tables:
 #                               Intel(r) ACPI Compiler from
 #                               https://acpica.org/downloads
+#
 #   CLANG35     -Linux,Windows-  Requires:
 #                             Clang v3.5 or later, and GNU binutils targeting aarch64-linux-gnu or arm-linux-gnueabi
 #                        Optional:
@@ -1682,9 +1683,9 @@ NOOPT_VS2017_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:428
 *_VS2017_ARM_ASLDLINK_PATH        = DEF(VS2017_BIN_ARM)\link.exe
 
       *_VS2017_ARM_MAKE_FLAGS     = /nologo
-  DEBUG_VS2017_ARM_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Gw /Oi-
+  DEBUG_VS2017_ARM_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gw /Oi-
 RELEASE_VS2017_ARM_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /Oi-
-NOOPT_VS2017_ARM_CC_FLAGS         = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od /Oi-
+NOOPT_VS2017_ARM_CC_FLAGS         = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Od /Oi-
 
   DEBUG_VS2017_ARM_ASM_FLAGS      = /nologo /g
 RELEASE_VS2017_ARM_ASM_FLAGS      = /nologo
@@ -1709,9 +1710,9 @@ NOOPT_VS2017_ARM_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2017_AARCH64_ASLDLINK_PATH     = DEF(VS2017_BIN_AARCH64)\link.exe
 
       *_VS2017_AARCH64_MAKE_FLAGS  = /nologo
-  DEBUG_VS2017_AARCH64_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Gw /Oi-
+  DEBUG_VS2017_AARCH64_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gw /Oi-
 RELEASE_VS2017_AARCH64_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /Oi-
-NOOPT_VS2017_AARCH64_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od /Oi-
+NOOPT_VS2017_AARCH64_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Od /Oi-
 
   DEBUG_VS2017_AARCH64_ASM_FLAGS   = /nologo /g
 RELEASE_VS2017_AARCH64_ASM_FLAGS   = /nologo
@@ -1845,9 +1846,9 @@ NOOPT_VS2019_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:428
 *_VS2019_ARM_ASLDLINK_PATH        = DEF(VS2019_BIN_ARM)\link.exe
 
       *_VS2019_ARM_MAKE_FLAGS     = /nologo
-  DEBUG_VS2019_ARM_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Gw /Oi-
+  DEBUG_VS2019_ARM_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gw /Oi-
 RELEASE_VS2019_ARM_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /Oi-
-NOOPT_VS2019_ARM_CC_FLAGS         = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od /Oi-
+NOOPT_VS2019_ARM_CC_FLAGS         = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Od /Oi-
 
   DEBUG_VS2019_ARM_ASM_FLAGS      = /nologo /g
 RELEASE_VS2019_ARM_ASM_FLAGS      = /nologo
@@ -1872,9 +1873,9 @@ NOOPT_VS2019_ARM_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2019_AARCH64_ASLDLINK_PATH     = DEF(VS2019_BIN_AARCH64)\link.exe
 
       *_VS2019_AARCH64_MAKE_FLAGS  = /nologo
-  DEBUG_VS2019_AARCH64_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Gw /Oi-
+  DEBUG_VS2019_AARCH64_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gw /Oi-
 RELEASE_VS2019_AARCH64_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /Oi-
-NOOPT_VS2019_AARCH64_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od /Oi-
+NOOPT_VS2019_AARCH64_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Od /Oi-
 
   DEBUG_VS2019_AARCH64_ASM_FLAGS   = /nologo /g
 RELEASE_VS2019_AARCH64_ASM_FLAGS   = /nologo
@@ -1946,6 +1947,7 @@ DEFINE GCC_IA32_RC_FLAGS           = -I binary -O elf32-i386          -B i386
 DEFINE GCC_X64_RC_FLAGS            = -I binary -O elf64-x86-64        -B i386    --rename-section .data=.hii
 DEFINE GCC_ARM_RC_FLAGS            = -I binary -O elf32-littlearm     -B arm     --rename-section .data=.hii
 DEFINE GCC_AARCH64_RC_FLAGS        = -I binary -O elf64-littleaarch64 -B aarch64 --rename-section .data=.hii
+DEFINE GCC_RISCV64_RC_FLAGS        = -I binary -O elf64-littleriscv   -B riscv   --rename-section .data=.hii
 
 # GCC Build Flag for included header file list generation
 DEFINE GCC_DEPS_FLAGS              = -MMD -MF $@.deps
@@ -2020,6 +2022,22 @@ DEFINE GCC5_ARM_ASLDLINK_FLAGS       = DEF(GCC49_ARM_ASLDLINK_FLAGS)
 DEFINE GCC5_AARCH64_ASLDLINK_FLAGS   = DEF(GCC49_AARCH64_ASLDLINK_FLAGS)
 DEFINE GCC5_ASLCC_FLAGS              = DEF(GCC49_ASLCC_FLAGS) -fno-lto
 
+DEFINE GCC5_RISCV_ALL_CC_FLAGS                    = -g -fshort-wchar -fno-strict-aliasing -Wall -Werror -Wno-array-bounds -ffunction-sections -fdata-sections -include AutoGen.h -fno-common -DSTRING_ARRAY_NAME=$(BASE_NAME)Strings -msmall-data-limit=0
+DEFINE GCC5_RISCV_ALL_DLINK_COMMON                = -nostdlib -Wl,-n,-q,--gc-sections -z common-page-size=0x40
+DEFINE GCC5_RISCV_ALL_DLINK_FLAGS                 = DEF(GCC5_RISCV_ALL_DLINK_COMMON) -Wl,--entry,$(IMAGE_ENTRY_POINT) -u $(IMAGE_ENTRY_POINT) -Wl,-Map,$(DEST_DIR_DEBUG)/$(BASE_NAME).map
+DEFINE GCC5_RISCV_ALL_DLINK2_FLAGS                = -Wl,--defsym=PECOFF_HEADER_SIZE=0x220,--script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
+DEFINE GCC5_RISCV_ALL_ASM_FLAGS                   = -c -x assembler -imacros $(DEST_DIR_DEBUG)/AutoGen.h
+DEFINE GCC5_RISCV_ALL_CC_FLAGS_WARNING_DISABLE    = -Wno-tautological-compare -Wno-pointer-compare
+
+DEFINE GCC5_RISCV_OPENSBI_TYPES                   = -DOPENSBI_EXTERNAL_SBI_TYPES=OpensbiTypes.h
+
+DEFINE GCC5_RISCV64_ARCH                   = rv64imafdc
+DEFINE GCC5_RISCV32_RISCV64_ASLDLINK_FLAGS = DEF(GCC5_RISCV_ALL_DLINK_COMMON) -Wl,--entry,ReferenceAcpiTable -u ReferenceAcpiTable
+DEFINE GCC5_RISCV64_CC_FLAGS               = DEF(GCC5_RISCV_ALL_CC_FLAGS) DEF(GCC5_RISCV_ALL_CC_FLAGS_WARNING_DISABLE) DEF(GCC5_RISCV_OPENSBI_TYPES) -march=DEF(GCC5_RISCV64_ARCH) -fno-builtin -fno-builtin-memcpy -fno-stack-protector -Wno-address -fno-asynchronous-unwind-tables -Wno-unused-but-set-variable -fpack-struct=8 -mcmodel=medany -mabi=lp64 -mno-relax
+DEFINE GCC5_RISCV64_DLINK_FLAGS            = DEF(GCC5_RISCV_ALL_DLINK_FLAGS) -Wl,-melf64lriscv,--oformat=elf64-littleriscv,--no-relax
+DEFINE GCC5_RISCV64_DLINK2_FLAGS           = DEF(GCC5_RISCV_ALL_DLINK2_FLAGS)
+DEFINE GCC5_RISCV64_ASM_FLAGS              = DEF(GCC5_RISCV_ALL_ASM_FLAGS) -march=DEF(GCC5_RISCV64_ARCH) -mcmodel=medany -mabi=lp64
+
 ####################################################################################
 #
 # GCC 4.8 - This configuration is used to compile under Linux to produce
@@ -2464,6 +2482,37 @@ RELEASE_GCC5_AARCH64_DLINK_XIPFLAGS = -z common-page-size=0x20
   NOOPT_GCC5_AARCH64_DLINK_FLAGS = DEF(GCC5_AARCH64_DLINK_FLAGS) -O0
   NOOPT_GCC5_AARCH64_DLINK_XIPFLAGS = -z common-page-size=0x20 -O0
 
+####################################################################################
+#
+# GCC RISC-V This configuration is used to compile under Linux to produce
+#             PE/COFF binaries using GCC RISC-V tool chain
+#
+####################################################################################
+
+##################
+# GCC5 RISCV64 definitions
+##################
+*_GCC5_RISCV64_OBJCOPY_PATH         = ENV(GCC5_RISCV64_PREFIX)objcopy
+*_GCC5_RISCV64_CC_PATH              = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_SLINK_PATH           = ENV(GCC5_RISCV64_PREFIX)gcc-ar
+*_GCC5_RISCV64_DLINK_PATH           = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_ASLDLINK_PATH        = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_ASM_PATH             = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_PP_PATH              = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_VFRPP_PATH           = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_ASLCC_PATH           = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_ASLPP_PATH           = ENV(GCC5_RISCV64_PREFIX)gcc
+*_GCC5_RISCV64_RC_PATH              = ENV(GCC5_RISCV64_PREFIX)objcopy
+
+*_GCC5_RISCV64_ASLCC_FLAGS          = DEF(GCC_ASLCC_FLAGS)
+*_GCC5_RISCV64_ASLDLINK_FLAGS       = DEF(GCC5_RISCV32_RISCV64_ASLDLINK_FLAGS)
+*_GCC5_RISCV64_ASM_FLAGS            = DEF(GCC5_RISCV64_ASM_FLAGS)
+*_GCC5_RISCV64_CC_FLAGS             = DEF(GCC5_RISCV64_CC_FLAGS) -save-temps
+*_GCC5_RISCV64_DLINK_FLAGS          = DEF(GCC5_RISCV64_DLINK_FLAGS)
+*_GCC5_RISCV64_DLINK2_FLAGS         = DEF(GCC5_RISCV64_DLINK2_FLAGS)
+*_GCC5_RISCV64_RC_FLAGS             = DEF(GCC_RISCV64_RC_FLAGS)
+*_GCC5_RISCV64_OBJCOPY_FLAGS        =
+
 ####################################################################################
 #
 # CLANG35   - This configuration is used to compile under Linux to produce
@@ -2759,7 +2808,7 @@ DEFINE CLANGPDB_IA32_TARGET          = -target i686-unknown-windows-gnu
 DEFINE CLANGPDB_X64_TARGET           = -target x86_64-unknown-windows-gnu
 
 DEFINE CLANGPDB_WARNING_OVERRIDES    = -Wno-parentheses-equality -Wno-tautological-compare -Wno-tautological-constant-out-of-range-compare -Wno-empty-body -Wno-unused-const-variable -Wno-varargs -Wno-unknown-warning-option -Wno-microsoft-enum-forward-reference
-DEFINE CLANGPDB_ALL_CC_FLAGS         = DEF(GCC48_ALL_CC_FLAGS) DEF(CLANGPDB_WARNING_OVERRIDES) -fno-stack-protector -funsigned-char -ftrap-function=undefined_behavior_has_been_optimized_away_by_clang -Wno-address -Wno-shift-negative-value -Wno-unknown-pragmas -Wno-incompatible-library-redeclaration -Wno-null-dereference -mno-implicit-float -mms-bitfields -mno-stack-arg-probe -nostdlib -nostdlibinc
+DEFINE CLANGPDB_ALL_CC_FLAGS         = DEF(GCC48_ALL_CC_FLAGS) DEF(CLANGPDB_WARNING_OVERRIDES) -fno-stack-protector -funsigned-char -ftrap-function=undefined_behavior_has_been_optimized_away_by_clang -Wno-address -Wno-shift-negative-value -Wno-unknown-pragmas -Wno-incompatible-library-redeclaration -Wno-null-dereference -mno-implicit-float -mms-bitfields -mno-stack-arg-probe -nostdlib -nostdlibinc -fseh-exceptions
 
 ###########################
 # CLANGPDB IA32 definitions
@@ -2784,15 +2833,15 @@ DEFINE CLANGPDB_ALL_CC_FLAGS         = DEF(GCC48_ALL_CC_FLAGS) DEF(CLANGPDB_WARN
 *_CLANGPDB_IA32_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANGPDB_IA32_TARGET)
 
 DEBUG_CLANGPDB_IA32_CC_FLAGS         = DEF(CLANGPDB_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANGPDB_IA32_TARGET) -gcodeview
-DEBUG_CLANGPDB_IA32_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH  /lldmap
+DEBUG_CLANGPDB_IA32_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH /MLLVM:-exception-model=wineh /lldmap
 DEBUG_CLANGPDB_IA32_DLINK2_FLAGS     =
 
 RELEASE_CLANGPDB_IA32_CC_FLAGS       = DEF(CLANGPDB_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANGPDB_IA32_TARGET)
-RELEASE_CLANGPDB_IA32_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /MERGE:.rdata=.data  /lldmap
+RELEASE_CLANGPDB_IA32_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /MERGE:.rdata=.data /MLLVM:-exception-model=wineh /lldmap
 RELEASE_CLANGPDB_IA32_DLINK2_FLAGS   =
 
 NOOPT_CLANGPDB_IA32_CC_FLAGS         = DEF(CLANGPDB_ALL_CC_FLAGS) -m32 -O0 -march=i586 DEF(CLANGPDB_IA32_TARGET) -gcodeview
-NOOPT_CLANGPDB_IA32_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH  /lldmap
+NOOPT_CLANGPDB_IA32_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH /MLLVM:-exception-model=wineh /lldmap
 NOOPT_CLANGPDB_IA32_DLINK2_FLAGS     =
 
 ##########################
@@ -2818,17 +2867,17 @@ NOOPT_CLANGPDB_IA32_DLINK2_FLAGS     =
 *_CLANGPDB_X64_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANGPDB_X64_TARGET)
 
 DEBUG_CLANGPDB_X64_CC_FLAGS         = DEF(CLANGPDB_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -Oz -flto DEF(CLANGPDB_X64_TARGET) -gcodeview  -funwind-tables
-DEBUG_CLANGPDB_X64_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /Machine:X64 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH  /lldmap
+DEBUG_CLANGPDB_X64_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /Machine:X64 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH /MLLVM:-exception-model=wineh /lldmap
 DEBUG_CLANGPDB_X64_DLINK2_FLAGS     =
 DEBUG_CLANGPDB_X64_GENFW_FLAGS      = --keepexceptiontable
 
 RELEASE_CLANGPDB_X64_CC_FLAGS       = DEF(CLANGPDB_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -Oz -flto DEF(CLANGPDB_X64_TARGET) -fno-unwind-tables
-RELEASE_CLANGPDB_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /MERGE:.rdata=.data  /lldmap
+RELEASE_CLANGPDB_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /MERGE:.rdata=.data /MLLVM:-exception-model=wineh /lldmap
 RELEASE_CLANGPDB_X64_DLINK2_FLAGS   =
 RELEASE_CLANGPDB_X64_GENFW_FLAGS    =
 
 NOOPT_CLANGPDB_X64_CC_FLAGS         = DEF(CLANGPDB_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -O0 DEF(CLANGPDB_X64_TARGET) -gcodeview -funwind-tables
-NOOPT_CLANGPDB_X64_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /Machine:X64 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH  /lldmap
+NOOPT_CLANGPDB_X64_DLINK_FLAGS      = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /ALIGN:32 /FILEALIGN:32 /Machine:X64 /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DEBUG:GHASH /MLLVM:-exception-model=wineh /lldmap
 NOOPT_CLANGPDB_X64_DLINK2_FLAGS     =
 NOOPT_CLANGPDB_X64_GENFW_FLAGS      = --keepexceptiontable
 

BaseTools/Edk2ToolsBuild.py

@@ -11,6 +11,7 @@ import os
 import sys
 import logging
 import argparse
+import multiprocessing
 from edk2toolext import edk2_logging
 from edk2toolext.environment import self_describing_environment
 from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
@@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
             return ret
 
         elif self.tool_chain_tag.lower().startswith("gcc"):
-            ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+            cpu_count = self.GetCpuThreads()
+            ret = RunCmd("make", f"-C .  -j {cpu_count}", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
             if ret != 0:
                 raise Exception("Failed to build.")
 
@@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
         logging.critical("Tool Chain not supported")
         return -1
 
+    def GetCpuThreads(self) -> int:
+        ''' Function to return number of cpus. If error return 1'''
+        cpus = 1
+        try:
+            cpus = multiprocessing.cpu_count()
+        except:
+            # from the internet there are cases where cpu_count is not implemented.
+            # will handle error by just doing single proc build
+            pass
+        return cpus
+
+
 
 def main():
     Edk2ToolsBuild().Invoke()

BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py

@@ -8,47 +8,23 @@
 import os
 import logging
 import glob
+import stat
 import xml.etree.ElementTree
 from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
 from edk2toolext import edk2_logging
 import edk2toollib.windows.locate_tools as locate_tools
 from edk2toolext.environment import shell_environment
 from edk2toollib.utility_functions import RunCmd
+from edk2toollib.utility_functions import GetHostInfo
 
 
 class HostBasedUnitTestRunner(IUefiBuildPlugin):
 
     def do_pre_build(self, thebuilder):
         '''
-        Works with the compiler (either the HostBasedCompilerPlugin or an other Builder) to set
-        up the environment that will be needed to build host-based unit tests.
-
-        EXPECTS:
-        - Build Var 'CI_BUILD_TYPE' - If not set to 'host_unit_test', will not do anything.
-
-        UPDATES:
-        - Shell Var (Several) - Updates the shell with all vars listed in interesting_keys.
-        - Shell Path - Updated from QueryVcVariables()
-        - Shell Var 'CMOCKA_MESSAGE_OUTPUT'
+        Run Prebuild
         '''
-        ci_type = thebuilder.env.GetValue('CI_BUILD_TYPE')
-        if ci_type != 'host_unit_test':
-            return 0
 
-        shell_env = shell_environment.GetEnvironment()
-        # Use the tools lib to determine the correct values for the vars that interest us.
-        interesting_keys = ["ExtensionSdkDir", "INCLUDE", "LIB", "LIBPATH", "UniversalCRTSdkDir",
-                            "UCRTVersion", "WindowsLibPath", "WindowsSdkBinPath", "WindowsSdkDir", "WindowsSdkVerBinPath",
-                            "WindowsSDKVersion", "VCToolsInstallDir"]
-        vs_vars = locate_tools.QueryVcVariables(interesting_keys, "amd64")
-        for (k, v) in vs_vars.items():
-            if k.upper() == "PATH":
-                shell_env.append_path(v)
-            else:
-                shell_env.set_shell_var(k, v)
-
-        # Set up the reporting type for Cmocka.
-        shell_env.set_shell_var('CMOCKA_MESSAGE_OUTPUT', 'xml')
         return 0
 
     def do_post_build(self, thebuilder):
@@ -73,6 +49,9 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin):
 
         failure_count = 0
 
+        # Set up the reporting type for Cmocka.
+        shell_env.set_shell_var('CMOCKA_MESSAGE_OUTPUT', 'xml')
+
         for arch in thebuilder.env.GetValue("TARGET_ARCH").split():
             logging.log(edk2_logging.get_subsection_level(),
                         "Testing for architecture: " + arch)
@@ -82,8 +61,29 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin):
             for old_result in glob.iglob(os.path.join(cp, "*.result.xml")):
                 os.remove(old_result)
 
-            # Determine whether any tests exist.
-            testList = glob.glob(os.path.join(cp, "*Test*.exe"))
+            # Find and Run any Host Tests
+            if GetHostInfo().os.upper() == "LINUX":
+                testList = glob.glob(os.path.join(cp, "*Test*"))
+                for a in testList[:]:
+                    p = os.path.join(cp, a)
+                    # It must be a file
+                    if not os.path.isfile(p):
+                        testList.remove(a)
+                        logging.debug(f"Remove directory file: {p}")
+                        continue
+                    # It must be executable
+                    if os.stat(p).st_mode & (stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH) == 0:
+                        testList.remove(a)
+                        logging.debug(f"Remove non-executable file: {p}")
+                        continue
+
+                    logging.info(f"Test file found: {p}")
+
+            elif GetHostInfo().os.upper() == "WINDOWS":
+                testList = glob.glob(os.path.join(cp, "*Test*.exe"))
+            else:
+                raise NotImplementedError("Unsupported Operating System")
+
             for test in testList:
                 # Configure output name.
                 shell_env.set_shell_var(

BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner_plug_in.yaml

@@ -6,7 +6,7 @@
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 ##
 {
-  "scope": "host-test-win",
-  "name": "Windows Host-Based Unit Test Runner",
+  "scope": "host-based-test",
+  "name": "Host-Based Unit Test Runner",
   "module": "HostBasedUnitTestRunner"
 }

BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py

@@ -4,6 +4,7 @@
 # This plugin works in conjuncture with the tools_def
 #
 # Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 ##
 import os
@@ -36,6 +37,12 @@ class LinuxGcc5ToolChain(IUefiBuildPlugin):
                 self.Logger.critical("Failed in check arm")
                 return ret
 
+            # Check RISCV64 compiler
+            ret = self._check_riscv64()
+            if ret != 0:
+                self.Logger.critical("Failed in check riscv64")
+                return ret
+
         return 0
 
     def _check_arm(self):
@@ -83,3 +90,34 @@ class LinuxGcc5ToolChain(IUefiBuildPlugin):
             return -2
 
         return 0
+
+    def _check_riscv64(self):
+        # now check for install dir.  If set then set the Prefix
+        install_path = shell_environment.GetEnvironment(
+        ).get_shell_var("GCC5_RISCV64_INSTALL")
+        if install_path is None:
+            return 0
+
+        # check to see if full path already configured
+        if shell_environment.GetEnvironment().get_shell_var("GCC5_RISCV64_PREFIX") is not None:
+            self.Logger.info("GCC5_RISCV64_PREFIX is already set.")
+
+        else:
+            # make GCC5_RISCV64_PREFIX to align with tools_def.txt
+            prefix = os.path.join(install_path, "bin", "riscv64-unknown-elf-")
+            shell_environment.GetEnvironment().set_shell_var("GCC5_RISCV64_PREFIX", prefix)
+
+        # now confirm it exists
+        if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("GCC5_RISCV64_PREFIX") + "gcc"):
+            self.Logger.error(
+                "Path for GCC5_RISCV64_PREFIX toolchain is invalid")
+            return -2
+
+        # Check if LD_LIBRARY_PATH is set for the libraries of RISC-V GCC toolchain
+        if shell_environment.GetEnvironment().get_shell_var("LD_LIBRARY_PATH") is not None:
+            self.Logger.info("LD_LIBRARY_PATH is already set.")
+
+        prefix = os.path.join(install_path, "lib")
+        shell_environment.GetEnvironment().set_shell_var("LD_LIBRARY_PATH", prefix)
+
+        return 0

BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py

@@ -13,6 +13,7 @@ import edk2toollib.windows.locate_tools as locate_tools
 from edk2toollib.windows.locate_tools import FindWithVsWhere
 from edk2toolext.environment import shell_environment
 from edk2toolext.environment import version_aggregator
+from edk2toollib.utility_functions import GetHostInfo
 
 
 class WindowsVsToolChain(IUefiBuildPlugin):
@@ -26,14 +27,41 @@ class WindowsVsToolChain(IUefiBuildPlugin):
                             "UCRTVersion", "WindowsLibPath", "WindowsSdkBinPath", "WindowsSdkDir", "WindowsSdkVerBinPath",
                             "WindowsSDKVersion", "VCToolsInstallDir", "Path"]
 
-#
+        #
         # VS2017 - Follow VS2017 where there is potential for many versions of the tools.
         # If a specific version is required then the user must set both env variables:
         # VS150INSTALLPATH:  base install path on system to VC install dir.  Here you will find the VC folder, etc
         # VS150TOOLVER:      version number for the VC compiler tools
         # VS2017_PREFIX:     path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+        # VS2017_HOST:       set the host architecture to use for host tools, and host libs, etc
         if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2017":
 
+            # check to see if host is configured
+            # HostType for VS2017 should be (defined in tools_def):
+            # x86   == 32bit Intel
+            # x64   == 64bit Intel
+            # arm   == 32bit Arm
+            # arm64 == 64bit Arm
+            #
+            HostType = shell_environment.GetEnvironment().get_shell_var("VS2017_HOST")
+            if HostType is not None:
+                HostType = HostType.lower()
+                self.Logger.info(
+                    f"HOST TYPE defined by environment.  Host Type is {HostType}")
+            else:
+                HostInfo = GetHostInfo()
+                if HostInfo.arch == "x86":
+                    if HostInfo.bit == "32":
+                        HostType = "x86"
+                    elif HostInfo.bit == "64":
+                        HostType = "x64"
+                else:
+                    raise NotImplementedError()
+
+            # VS2017_HOST options are not exactly the same as QueryVcVariables. This translates.
+            VC_HOST_ARCH_TRANSLATOR = {
+                "x86": "x86", "x64": "AMD64", "arm": "not supported", "arm64": "not supported"}
+
             # check to see if full path already configured
             if shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX") != None:
                 self.Logger.info("VS2017_PREFIX is already set.")
@@ -58,16 +86,14 @@ class WindowsVsToolChain(IUefiBuildPlugin):
                                       "Tools", "MSVC", vc_ver)
                 prefix = prefix + os.path.sep
                 shell_environment.GetEnvironment().set_shell_var("VS2017_PREFIX", prefix)
+                shell_environment.GetEnvironment().set_shell_var("VS2017_HOST", HostType)
 
                 shell_env = shell_environment.GetEnvironment()
                 # Use the tools lib to determine the correct values for the vars that interest us.
                 vs_vars = locate_tools.QueryVcVariables(
-                    interesting_keys, "amd64", vs_version="vs2017")
+                    interesting_keys, VC_HOST_ARCH_TRANSLATOR[HostType], vs_version="vs2017")
                 for (k, v) in vs_vars.items():
-                    if k.upper() == "PATH":
-                        shell_env.insert_path(v)
-                    else:
-                        shell_env.set_shell_var(k, v)
+                    shell_env.set_shell_var(k, v)
 
             # now confirm it exists
             if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX")):
@@ -80,8 +106,35 @@ class WindowsVsToolChain(IUefiBuildPlugin):
         # VS160INSTALLPATH:  base install path on system to VC install dir.  Here you will find the VC folder, etc
         # VS160TOOLVER:      version number for the VC compiler tools
         # VS2019_PREFIX:     path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+        # VS2017_HOST:       set the host architecture to use for host tools, and host libs, etc
         elif thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2019":
 
+            # check to see if host is configured
+            # HostType for VS2019 should be (defined in tools_def):
+            # x86   == 32bit Intel
+            # x64   == 64bit Intel
+            # arm   == 32bit Arm
+            # arm64 == 64bit Arm
+            #
+            HostType = shell_environment.GetEnvironment().get_shell_var("VS2019_HOST")
+            if HostType is not None:
+                HostType = HostType.lower()
+                self.Logger.info(
+                    f"HOST TYPE defined by environment.  Host Type is {HostType}")
+            else:
+                HostInfo = GetHostInfo()
+                if HostInfo.arch == "x86":
+                    if HostInfo.bit == "32":
+                        HostType = "x86"
+                    elif HostInfo.bit == "64":
+                        HostType = "x64"
+                else:
+                    raise NotImplementedError()
+
+            # VS2019_HOST options are not exactly the same as QueryVcVariables. This translates.
+            VC_HOST_ARCH_TRANSLATOR = {
+                "x86": "x86", "x64": "AMD64", "arm": "not supported", "arm64": "not supported"}
+
             # check to see if full path already configured
             if shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX") != None:
                 self.Logger.info("VS2019_PREFIX is already set.")
@@ -106,16 +159,14 @@ class WindowsVsToolChain(IUefiBuildPlugin):
                                       "Tools", "MSVC", vc_ver)
                 prefix = prefix + os.path.sep
                 shell_environment.GetEnvironment().set_shell_var("VS2019_PREFIX", prefix)
+                shell_environment.GetEnvironment().set_shell_var("VS2019_HOST", HostType)
 
                 shell_env = shell_environment.GetEnvironment()
                 # Use the tools lib to determine the correct values for the vars that interest us.
                 vs_vars = locate_tools.QueryVcVariables(
-                    interesting_keys, "amd64", vs_version="vs2019")
+                    interesting_keys, VC_HOST_ARCH_TRANSLATOR[HostType], vs_version="vs2019")
                 for (k, v) in vs_vars.items():
-                    if k.upper() == "PATH":
-                        shell_env.insert_path(v)
-                    else:
-                        shell_env.set_shell_var(k, v)
+                    shell_env.set_shell_var(k, v)
 
             # now confirm it exists
             if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX")):

BaseTools/Scripts/GetMaintainer.py

@@ -77,8 +77,12 @@ def get_section_maintainers(path, section):
        matching the provided path in the provided section."""
     maintainers = []
     lists = []
+    nowarn_status = ['Supported', 'Maintained']
 
     if path_in_section(path, section):
+        for status in section['status']:
+            if status not in nowarn_status:
+                print('WARNING: Maintained status for "%s" is \'%s\'!' % (path, status))
         for address in section['maintainer'], section['reviewer']:
             # Convert to list if necessary
             if isinstance(address, list):

BaseTools/Scripts/SetupGit.py

@@ -106,10 +106,11 @@ def fuzzy_match_repo_url(one, other):
     return False
 
 
-def get_upstream(url):
+def get_upstream(url, name):
     """Extracts the dict for the current repo origin."""
     for upstream in UPSTREAMS:
-        if fuzzy_match_repo_url(upstream['repo'], url):
+        if (fuzzy_match_repo_url(upstream['repo'], url) or
+                upstream['name'] == name):
             return upstream
     print("Unknown upstream '%s' - aborting!" % url)
     sys.exit(3)
@@ -143,6 +144,11 @@ if __name__ == '__main__':
                         help='overwrite existing settings conflicting with program defaults',
                         action='store_true',
                         required=False)
+    PARSER.add_argument('-n', '--name', type=str, metavar='repo',
+                        choices=['edk2', 'edk2-platforms', 'edk2-non-osi'],
+                        help='set the repo name to configure for, if not '
+                             'detected automatically',
+                        required=False)
     PARSER.add_argument('-v', '--verbose',
                         help='enable more detailed output',
                         action='store_true',
@@ -156,7 +162,7 @@ if __name__ == '__main__':
 
     URL = REPO.remotes.origin.url
 
-    UPSTREAM = get_upstream(URL)
+    UPSTREAM = get_upstream(URL, ARGS.name)
     if not UPSTREAM:
         print("Upstream '%s' unknown, aborting!" % URL)
         sys.exit(7)

BaseTools/Source/C/BrotliCompress/GNUmakefile

@@ -1,7 +1,7 @@
 ## @file
 # GNU/Linux makefile for 'Brotli' module build.
 #
-# Copyright (c) 2017 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2017 - 2020, Intel Corporation. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 MAKEROOT ?= ..
@@ -9,33 +9,33 @@ MAKEROOT ?= ..
 APPNAME = Brotli
 
 OBJECTS = \
-  tools/brotli.o \
-  common/dictionary.o \
-  common/transform.o \
-  dec/bit_reader.o \
-  dec/decode.o \
-  dec/huffman.o \
-  dec/state.o \
-  enc/backward_references.o \
-  enc/backward_references_hq.o \
-  enc/bit_cost.o \
-  enc/block_splitter.o \
-  enc/brotli_bit_stream.o \
-  enc/cluster.o \
-  enc/compress_fragment.o \
-  enc/compress_fragment_two_pass.o \
-  enc/dictionary_hash.o \
-  enc/encode.o \
-  enc/encoder_dict.o \
-  enc/entropy_encode.o \
-  enc/histogram.o \
-  enc/literal_cost.o \
-  enc/memory.o \
-  enc/metablock.o \
-  enc/static_dict.o \
-  enc/utf8_util.o
+  brotli/c/tools/brotli.o \
+  brotli/c/common/dictionary.o \
+  brotli/c/common/transform.o \
+  brotli/c/dec/bit_reader.o \
+  brotli/c/dec/decode.o \
+  brotli/c/dec/huffman.o \
+  brotli/c/dec/state.o \
+  brotli/c/enc/backward_references.o \
+  brotli/c/enc/backward_references_hq.o \
+  brotli/c/enc/bit_cost.o \
+  brotli/c/enc/block_splitter.o \
+  brotli/c/enc/brotli_bit_stream.o \
+  brotli/c/enc/cluster.o \
+  brotli/c/enc/compress_fragment.o \
+  brotli/c/enc/compress_fragment_two_pass.o \
+  brotli/c/enc/dictionary_hash.o \
+  brotli/c/enc/encode.o \
+  brotli/c/enc/encoder_dict.o \
+  brotli/c/enc/entropy_encode.o \
+  brotli/c/enc/histogram.o \
+  brotli/c/enc/literal_cost.o \
+  brotli/c/enc/memory.o \
+  brotli/c/enc/metablock.o \
+  brotli/c/enc/static_dict.o \
+  brotli/c/enc/utf8_util.o
 
 include $(MAKEROOT)/Makefiles/app.makefile
 
-TOOL_INCLUDE = -I ./include
+TOOL_INCLUDE = -I ./brotli/c/include
 LIBS += -lm

BaseTools/Source/C/BrotliCompress/LICENSE

@@ -1,19 +0,0 @@
-Copyright (c) 2009, 2010, 2013-2016 by the Brotli Authors.
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.

BaseTools/Source/C/BrotliCompress/Makefile

@@ -1,46 +1,46 @@
 ## @file
 # Windows makefile for 'Brotli' module build.
 #
-# Copyright (c) 2017 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2017 - 2020, Intel Corporation. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 !INCLUDE ..\Makefiles\ms.common
 
-INC = -I .\include $(INC)
+INC = -I .\brotli\c\include $(INC)
 CFLAGS = $(CFLAGS) /W2
 
 APPNAME = Brotli
 
 #LIBS = $(LIB_PATH)\Common.lib
 
-COMMON_OBJ = common\dictionary.obj common\transform.obj
+COMMON_OBJ = brotli\c\common\dictionary.obj brotli\c\common\transform.obj
 DEC_OBJ = \
-  dec\bit_reader.obj \
-  dec\decode.obj \
-  dec\huffman.obj \
-  dec\state.obj
+  brotli\c\dec\bit_reader.obj \
+  brotli\c\dec\decode.obj \
+  brotli\c\dec\huffman.obj \
+  brotli\c\dec\state.obj
 ENC_OBJ = \
-  enc\backward_references.obj \
-  enc\backward_references_hq.obj \
-  enc\bit_cost.obj \
-  enc\block_splitter.obj \
-  enc\brotli_bit_stream.obj \
-  enc\cluster.obj \
-  enc\compress_fragment.obj \
-  enc\compress_fragment_two_pass.obj \
-  enc\dictionary_hash.obj \
-  enc\encode.obj \
-  enc\encoder_dict.obj \
-  enc\entropy_encode.obj \
-  enc\histogram.obj \
-  enc\literal_cost.obj \
-  enc\memory.obj \
-  enc\metablock.obj \
-  enc\static_dict.obj \
-  enc\utf8_util.obj
+  brotli\c\enc\backward_references.obj \
+  brotli\c\enc\backward_references_hq.obj \
+  brotli\c\enc\bit_cost.obj \
+  brotli\c\enc\block_splitter.obj \
+  brotli\c\enc\brotli_bit_stream.obj \
+  brotli\c\enc\cluster.obj \
+  brotli\c\enc\compress_fragment.obj \
+  brotli\c\enc\compress_fragment_two_pass.obj \
+  brotli\c\enc\dictionary_hash.obj \
+  brotli\c\enc\encode.obj \
+  brotli\c\enc\encoder_dict.obj \
+  brotli\c\enc\entropy_encode.obj \
+  brotli\c\enc\histogram.obj \
+  brotli\c\enc\literal_cost.obj \
+  brotli\c\enc\memory.obj \
+  brotli\c\enc\metablock.obj \
+  brotli\c\enc\static_dict.obj \
+  brotli\c\enc\utf8_util.obj
 
 OBJECTS = \
-  tools\brotli.obj \
+  brotli\c\tools\brotli.obj \
   $(COMMON_OBJ) \
   $(DEC_OBJ) \
   $(ENC_OBJ)

BaseTools/Source/C/BrotliCompress/README.md

@@ -1,26 +0,0 @@
-### Introduction
-
-Brotli is a generic-purpose lossless compression algorithm that compresses data
-using a combination of a modern variant of the LZ77 algorithm, Huffman coding
-and 2nd order context modeling, with a compression ratio comparable to the best
-currently available general-purpose compression methods. It is similar in speed
-with deflate but offers more dense compression.
-
-The specification of the Brotli Compressed Data Format is defined in [RFC 7932](https://www.ietf.org/rfc/rfc7932.txt).
-
-Brotli is open-sourced under the MIT License, see the LICENSE file.
-
-Brotli mailing list:
-https://groups.google.com/forum/#!forum/brotli
-
-[![Build Status](https://travis-ci.org/google/brotli.svg?branch=master)](https://travis-ci.org/google/brotli)
-
-### Benchmarks
-* [Squash Compression Benchmark](https://quixdb.github.io/squash-benchmark/) / [Unstable Squash Compression Benchmark](https://quixdb.github.io/squash-benchmark/unstable/)
-* [Large Text Compression Benchmark](http://mattmahoney.net/dc/text.html)
-* [Lzturbo Benchmark](https://sites.google.com/site/powturbo/home/benchmark)
-
-### Related projects
-Independent [decoder](https://github.com/madler/brotli) implementation by Mark Adler, based entirely on format specification.
-
-JavaScript port of brotli [decoder](https://github.com/devongovett/brotli.js). Could be used directly via `npm install brotli`

BaseTools/Source/C/BrotliCompress/ReadMe.txt

@@ -1,2 +0,0 @@
-It is based on the Brotli v1.0.6.
-Brotli was released on the website https://github.com/google/brotli.

BaseTools/Source/C/BrotliCompress/common/constants.h

@@ -1,64 +0,0 @@
-/* Copyright 2016 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-#ifndef BROTLI_COMMON_CONSTANTS_H_
-#define BROTLI_COMMON_CONSTANTS_H_
-
-/* Specification: 7.3. Encoding of the context map */
-#define BROTLI_CONTEXT_MAP_MAX_RLE 16
-
-/* Specification: 2. Compressed representation overview */
-#define BROTLI_MAX_NUMBER_OF_BLOCK_TYPES 256
-
-/* Specification: 3.3. Alphabet sizes: insert-and-copy length */
-#define BROTLI_NUM_LITERAL_SYMBOLS 256
-#define BROTLI_NUM_COMMAND_SYMBOLS 704
-#define BROTLI_NUM_BLOCK_LEN_SYMBOLS 26
-#define BROTLI_MAX_CONTEXT_MAP_SYMBOLS (BROTLI_MAX_NUMBER_OF_BLOCK_TYPES + \
-                                        BROTLI_CONTEXT_MAP_MAX_RLE)
-#define BROTLI_MAX_BLOCK_TYPE_SYMBOLS (BROTLI_MAX_NUMBER_OF_BLOCK_TYPES + 2)
-
-/* Specification: 3.5. Complex prefix codes */
-#define BROTLI_REPEAT_PREVIOUS_CODE_LENGTH 16
-#define BROTLI_REPEAT_ZERO_CODE_LENGTH 17
-#define BROTLI_CODE_LENGTH_CODES (BROTLI_REPEAT_ZERO_CODE_LENGTH + 1)
-/* "code length of 8 is repeated" */
-#define BROTLI_INITIAL_REPEATED_CODE_LENGTH 8
-
-/* "Large Window Brotli" */
-#define BROTLI_LARGE_MAX_DISTANCE_BITS 62U
-#define BROTLI_LARGE_MIN_WBITS 10
-#define BROTLI_LARGE_MAX_WBITS 30
-
-/* Specification: 4. Encoding of distances */
-#define BROTLI_NUM_DISTANCE_SHORT_CODES 16
-#define BROTLI_MAX_NPOSTFIX 3
-#define BROTLI_MAX_NDIRECT 120
-#define BROTLI_MAX_DISTANCE_BITS 24U
-#define BROTLI_DISTANCE_ALPHABET_SIZE(NPOSTFIX, NDIRECT, MAXNBITS) ( \
-    BROTLI_NUM_DISTANCE_SHORT_CODES + (NDIRECT) +                    \
-    ((MAXNBITS) << ((NPOSTFIX) + 1)))
-/* BROTLI_NUM_DISTANCE_SYMBOLS == 1128 */
-#define BROTLI_NUM_DISTANCE_SYMBOLS \
-    BROTLI_DISTANCE_ALPHABET_SIZE(  \
-        BROTLI_MAX_NDIRECT, BROTLI_MAX_NPOSTFIX, BROTLI_LARGE_MAX_DISTANCE_BITS)
-#define BROTLI_MAX_DISTANCE 0x3FFFFFC
-#define BROTLI_MAX_ALLOWED_DISTANCE 0x7FFFFFFC
-
-/* 7.1. Context modes and context ID lookup for literals */
-/* "context IDs for literals are in the range of 0..63" */
-#define BROTLI_LITERAL_CONTEXT_BITS 6
-
-/* 7.2. Context ID for distances */
-#define BROTLI_DISTANCE_CONTEXT_BITS 2
-
-/* 9.1. Format of the Stream Header */
-/* Number of slack bytes for window size. Don't confuse
-   with BROTLI_NUM_DISTANCE_SHORT_CODES. */
-#define BROTLI_WINDOW_GAP 16
-#define BROTLI_MAX_BACKWARD_LIMIT(W) (((size_t)1 << (W)) - BROTLI_WINDOW_GAP)
-
-#endif  /* BROTLI_COMMON_CONSTANTS_H_ */

BaseTools/Source/C/BrotliCompress/common/context.h

@@ -1,261 +0,0 @@
-/* Copyright 2013 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Lookup table to map the previous two bytes to a context id.
-
-  There are four different context modeling modes defined here:
-    CONTEXT_LSB6: context id is the least significant 6 bits of the last byte,
-    CONTEXT_MSB6: context id is the most significant 6 bits of the last byte,
-    CONTEXT_UTF8: second-order context model tuned for UTF8-encoded text,
-    CONTEXT_SIGNED: second-order context model tuned for signed integers.
-
-  If |p1| and |p2| are the previous two bytes, and |mode| is current context
-  mode, we calculate the context as:
-
-    context = ContextLut(mode)[p1] | ContextLut(mode)[p2 + 256].
-
-  For CONTEXT_UTF8 mode, if the previous two bytes are ASCII characters
-  (i.e. < 128), this will be equivalent to
-
-    context = 4 * context1(p1) + context2(p2),
-
-  where context1 is based on the previous byte in the following way:
-
-    0  : non-ASCII control
-    1  : \t, \n, \r
-    2  : space
-    3  : other punctuation
-    4  : " '
-    5  : %
-    6  : ( < [ {
-    7  : ) > ] }
-    8  : , ; :
-    9  : .
-    10 : =
-    11 : number
-    12 : upper-case vowel
-    13 : upper-case consonant
-    14 : lower-case vowel
-    15 : lower-case consonant
-
-  and context2 is based on the second last byte:
-
-    0 : control, space
-    1 : punctuation
-    2 : upper-case letter, number
-    3 : lower-case letter
-
-  If the last byte is ASCII, and the second last byte is not (in a valid UTF8
-  stream it will be a continuation byte, value between 128 and 191), the
-  context is the same as if the second last byte was an ASCII control or space.
-
-  If the last byte is a UTF8 lead byte (value >= 192), then the next byte will
-  be a continuation byte and the context id is 2 or 3 depending on the LSB of
-  the last byte and to a lesser extent on the second last byte if it is ASCII.
-
-  If the last byte is a UTF8 continuation byte, the second last byte can be:
-    - continuation byte: the next byte is probably ASCII or lead byte (assuming
-      4-byte UTF8 characters are rare) and the context id is 0 or 1.
-    - lead byte (192 - 207): next byte is ASCII or lead byte, context is 0 or 1
-    - lead byte (208 - 255): next byte is continuation byte, context is 2 or 3
-
-  The possible value combinations of the previous two bytes, the range of
-  context ids and the type of the next byte is summarized in the table below:
-
-  |--------\-----------------------------------------------------------------|
-  |         \                         Last byte                              |
-  | Second   \---------------------------------------------------------------|
-  | last byte \    ASCII            |   cont. byte        |   lead byte      |
-  |            \   (0-127)          |   (128-191)         |   (192-)         |
-  |=============|===================|=====================|==================|
-  |  ASCII      | next: ASCII/lead  |  not valid          |  next: cont.     |
-  |  (0-127)    | context: 4 - 63   |                     |  context: 2 - 3  |
-  |-------------|-------------------|---------------------|------------------|
-  |  cont. byte | next: ASCII/lead  |  next: ASCII/lead   |  next: cont.     |
-  |  (128-191)  | context: 4 - 63   |  context: 0 - 1     |  context: 2 - 3  |
-  |-------------|-------------------|---------------------|------------------|
-  |  lead byte  | not valid         |  next: ASCII/lead   |  not valid       |
-  |  (192-207)  |                   |  context: 0 - 1     |                  |
-  |-------------|-------------------|---------------------|------------------|
-  |  lead byte  | not valid         |  next: cont.        |  not valid       |
-  |  (208-)     |                   |  context: 2 - 3     |                  |
-  |-------------|-------------------|---------------------|------------------|
-*/
-
-#ifndef BROTLI_COMMON_CONTEXT_H_
-#define BROTLI_COMMON_CONTEXT_H_
-
-#include <brotli/types.h>
-
-typedef enum ContextType {
-  CONTEXT_LSB6 = 0,
-  CONTEXT_MSB6 = 1,
-  CONTEXT_UTF8 = 2,
-  CONTEXT_SIGNED = 3
-} ContextType;
-
-/* Common context lookup table for all context modes. */
-static const uint8_t kContextLookup[2048] = {
-  /* CONTEXT_LSB6, last byte. */
-   0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
-  16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
-  32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
-  48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
-   0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
-  16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
-  32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
-  48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
-   0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
-  16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
-  32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
-  48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
-   0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
-  16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
-  32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
-  48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
-
-  /* CONTEXT_LSB6, second last byte, */
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-
-  /* CONTEXT_MSB6, last byte. */
-   0,  0,  0,  0,  1,  1,  1,  1,  2,  2,  2,  2,  3,  3,  3,  3,
-   4,  4,  4,  4,  5,  5,  5,  5,  6,  6,  6,  6,  7,  7,  7,  7,
-   8,  8,  8,  8,  9,  9,  9,  9, 10, 10, 10, 10, 11, 11, 11, 11,
-  12, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14, 14, 15, 15, 15, 15,
-  16, 16, 16, 16, 17, 17, 17, 17, 18, 18, 18, 18, 19, 19, 19, 19,
-  20, 20, 20, 20, 21, 21, 21, 21, 22, 22, 22, 22, 23, 23, 23, 23,
-  24, 24, 24, 24, 25, 25, 25, 25, 26, 26, 26, 26, 27, 27, 27, 27,
-  28, 28, 28, 28, 29, 29, 29, 29, 30, 30, 30, 30, 31, 31, 31, 31,
-  32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35,
-  36, 36, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 39, 39, 39,
-  40, 40, 40, 40, 41, 41, 41, 41, 42, 42, 42, 42, 43, 43, 43, 43,
-  44, 44, 44, 44, 45, 45, 45, 45, 46, 46, 46, 46, 47, 47, 47, 47,
-  48, 48, 48, 48, 49, 49, 49, 49, 50, 50, 50, 50, 51, 51, 51, 51,
-  52, 52, 52, 52, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 55, 55,
-  56, 56, 56, 56, 57, 57, 57, 57, 58, 58, 58, 58, 59, 59, 59, 59,
-  60, 60, 60, 60, 61, 61, 61, 61, 62, 62, 62, 62, 63, 63, 63, 63,
-
-  /* CONTEXT_MSB6, second last byte, */
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-
-  /* CONTEXT_UTF8, last byte. */
-  /* ASCII range. */
-   0,  0,  0,  0,  0,  0,  0,  0,  0,  4,  4,  0,  0,  4,  0,  0,
-   0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
-   8, 12, 16, 12, 12, 20, 12, 16, 24, 28, 12, 12, 32, 12, 36, 12,
-  44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 32, 32, 24, 40, 28, 12,
-  12, 48, 52, 52, 52, 48, 52, 52, 52, 48, 52, 52, 52, 52, 52, 48,
-  52, 52, 52, 52, 52, 48, 52, 52, 52, 52, 52, 24, 12, 28, 12, 12,
-  12, 56, 60, 60, 60, 56, 60, 60, 60, 56, 60, 60, 60, 60, 60, 56,
-  60, 60, 60, 60, 60, 56, 60, 60, 60, 60, 60, 24, 12, 28, 12,  0,
-  /* UTF8 continuation byte range. */
-  0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
-  0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
-  0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
-  0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
-  /* UTF8 lead byte range. */
-  2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
-  2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
-  2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
-  2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
-
-  /* CONTEXT_UTF8 second last byte. */
-  /* ASCII range. */
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1,
-  1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1,
-  1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
-  3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 0,
-  /* UTF8 continuation byte range. */
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  /* UTF8 lead byte range. */
-  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-
-  /* CONTEXT_SIGNED, last byte, same as the above values shifted by 3 bits. */
-   0, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
-  16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
-  16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
-  16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
-  24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-  24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-  24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-  24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-  32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
-  32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
-  32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
-  32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
-  40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
-  40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
-  40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
-  48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 56,
-
-  /* CONTEXT_SIGNED, second last byte. */
-  0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-  2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-  3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
-  3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
-  3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
-  3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
-  4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-  4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-  4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-  4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-  5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
-  5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
-  5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
-  6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7,
-};
-
-typedef const uint8_t* ContextLut;
-
-/* typeof(MODE) == ContextType; returns ContextLut */
-#define BROTLI_CONTEXT_LUT(MODE) (&kContextLookup[(MODE) << 9])
-
-/* typeof(LUT) == ContextLut */
-#define BROTLI_CONTEXT(P1, P2, LUT) ((LUT)[P1] | ((LUT) + 256)[P2])
-
-#endif  /* BROTLI_COMMON_CONTEXT_H_ */

BaseTools/Source/C/BrotliCompress/common/dictionary.h

@@ -1,64 +0,0 @@
-/* Copyright 2013 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Collection of static dictionary words. */
-
-#ifndef BROTLI_COMMON_DICTIONARY_H_
-#define BROTLI_COMMON_DICTIONARY_H_
-
-#include <brotli/port.h>
-#include <brotli/types.h>
-
-#if defined(__cplusplus) || defined(c_plusplus)
-extern "C" {
-#endif
-
-typedef struct BrotliDictionary {
-  /**
-   * Number of bits to encode index of dictionary word in a bucket.
-   *
-   * Specification: Appendix A. Static Dictionary Data
-   *
-   * Words in a dictionary are bucketed by length.
-   * @c 0 means that there are no words of a given length.
-   * Dictionary consists of words with length of [4..24] bytes.
-   * Values at [0..3] and [25..31] indices should not be addressed.
-   */
-  uint8_t size_bits_by_length[32];
-
-  /* assert(offset[i + 1] == offset[i] + (bits[i] ? (i << bits[i]) : 0)) */
-  uint32_t offsets_by_length[32];
-
-  /* assert(data_size == offsets_by_length[31]) */
-  size_t data_size;
-
-  /* Data array is not bound, and should obey to size_bits_by_length values.
-     Specified size matches default (RFC 7932) dictionary. Its size is
-     defined by data_size */
-  const uint8_t* data;
-} BrotliDictionary;
-
-BROTLI_COMMON_API const BrotliDictionary* BrotliGetDictionary(void);
-
-/**
- * Sets dictionary data.
- *
- * When dictionary data is already set / present, this method is no-op.
- *
- * Dictionary data MUST be provided before BrotliGetDictionary is invoked.
- * This method is used ONLY in multi-client environment (e.g. C + Java),
- * to reduce storage by sharing single dictionary between implementations.
- */
-BROTLI_COMMON_API void BrotliSetDictionaryData(const uint8_t* data);
-
-#define BROTLI_MIN_DICTIONARY_WORD_LENGTH 4
-#define BROTLI_MAX_DICTIONARY_WORD_LENGTH 24
-
-#if defined(__cplusplus) || defined(c_plusplus)
-}  /* extern "C" */
-#endif
-
-#endif  /* BROTLI_COMMON_DICTIONARY_H_ */

BaseTools/Source/C/BrotliCompress/common/platform.h

@@ -1,558 +0,0 @@
-/* Copyright 2016 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Macros for compiler / platform specific features and build options.
-
-   Build options are:
-    * BROTLI_BUILD_32_BIT disables 64-bit optimizations
-    * BROTLI_BUILD_64_BIT forces to use 64-bit optimizations
-    * BROTLI_BUILD_BIG_ENDIAN forces to use big-endian optimizations
-    * BROTLI_BUILD_ENDIAN_NEUTRAL disables endian-aware optimizations
-    * BROTLI_BUILD_LITTLE_ENDIAN forces to use little-endian optimizations
-    * BROTLI_BUILD_PORTABLE disables dangerous optimizations, like unaligned
-      read and overlapping memcpy; this reduces decompression speed by 5%
-    * BROTLI_BUILD_NO_RBIT disables "rbit" optimization for ARM CPUs
-    * BROTLI_DEBUG dumps file name and line number when decoder detects stream
-      or memory error
-    * BROTLI_ENABLE_LOG enables asserts and dumps various state information
-*/
-
-#ifndef BROTLI_COMMON_PLATFORM_H_
-#define BROTLI_COMMON_PLATFORM_H_
-
-#include <string.h>  /* memcpy */
-#include <stdlib.h>  /* malloc, free */
-
-#include <brotli/port.h>
-#include <brotli/types.h>
-
-#if defined(OS_LINUX) || defined(OS_CYGWIN)
-#include <endian.h>
-#elif defined(OS_FREEBSD)
-#include <machine/endian.h>
-#elif defined(OS_MACOSX)
-#include <machine/endian.h>
-/* Let's try and follow the Linux convention */
-#define BROTLI_X_BYTE_ORDER BYTE_ORDER
-#define BROTLI_X_LITTLE_ENDIAN LITTLE_ENDIAN
-#define BROTLI_X_BIG_ENDIAN BIG_ENDIAN
-#endif
-
-#if defined(BROTLI_ENABLE_LOG) || defined(BROTLI_DEBUG)
-#include <assert.h>
-#include <stdio.h>
-#endif
-
-/* The following macros were borrowed from https://github.com/nemequ/hedley
- * with permission of original author - Evan Nemerson <evan@nemerson.com> */
-
-/* >>> >>> >>> hedley macros */
-
-/* Define "BROTLI_PREDICT_TRUE" and "BROTLI_PREDICT_FALSE" macros for capable
-   compilers.
-
-To apply compiler hint, enclose the branching condition into macros, like this:
-
-  if (BROTLI_PREDICT_TRUE(zero == 0)) {
-    // main execution path
-  } else {
-    // compiler should place this code outside of main execution path
-  }
-
-OR:
-
-  if (BROTLI_PREDICT_FALSE(something_rare_or_unexpected_happens)) {
-    // compiler should place this code outside of main execution path
-  }
-
-*/
-#if BROTLI_GNUC_HAS_BUILTIN(__builtin_expect, 3, 0, 0) || \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0) ||               \
-    BROTLI_SUNPRO_VERSION_CHECK(5, 12, 0) ||              \
-    BROTLI_ARM_VERSION_CHECK(4, 1, 0) ||                  \
-    BROTLI_IBM_VERSION_CHECK(10, 1, 0) ||                 \
-    BROTLI_TI_VERSION_CHECK(7, 3, 0) ||                   \
-    BROTLI_TINYC_VERSION_CHECK(0, 9, 27)
-#define BROTLI_PREDICT_TRUE(x) (__builtin_expect(!!(x), 1))
-#define BROTLI_PREDICT_FALSE(x) (__builtin_expect(x, 0))
-#else
-#define BROTLI_PREDICT_FALSE(x) (x)
-#define BROTLI_PREDICT_TRUE(x) (x)
-#endif
-
-#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) && \
-    !defined(__cplusplus)
-#define BROTLI_RESTRICT restrict
-#elif BROTLI_GNUC_VERSION_CHECK(3, 1, 0) ||                         \
-    BROTLI_MSVC_VERSION_CHECK(14, 0, 0) ||                          \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0) ||                         \
-    BROTLI_ARM_VERSION_CHECK(4, 1, 0) ||                            \
-    BROTLI_IBM_VERSION_CHECK(10, 1, 0) ||                           \
-    BROTLI_PGI_VERSION_CHECK(17, 10, 0) ||                          \
-    BROTLI_TI_VERSION_CHECK(8, 0, 0) ||                             \
-    BROTLI_IAR_VERSION_CHECK(8, 0, 0) ||                            \
-    (BROTLI_SUNPRO_VERSION_CHECK(5, 14, 0) && defined(__cplusplus))
-#define BROTLI_RESTRICT __restrict
-#elif BROTLI_SUNPRO_VERSION_CHECK(5, 3, 0) && !defined(__cplusplus)
-#define BROTLI_RESTRICT _Restrict
-#else
-#define BROTLI_RESTRICT
-#endif
-
-#if (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)) || \
-    (defined(__cplusplus) && (__cplusplus >= 199711L))
-#define BROTLI_MAYBE_INLINE inline
-#elif defined(__GNUC_STDC_INLINE__) || defined(__GNUC_GNU_INLINE__) || \
-    BROTLI_ARM_VERSION_CHECK(6, 2, 0)
-#define BROTLI_MAYBE_INLINE __inline__
-#elif BROTLI_MSVC_VERSION_CHECK(12, 0, 0) || \
-    BROTLI_ARM_VERSION_CHECK(4, 1, 0) || BROTLI_TI_VERSION_CHECK(8, 0, 0)
-#define BROTLI_MAYBE_INLINE __inline
-#else
-#define BROTLI_MAYBE_INLINE
-#endif
-
-#if BROTLI_GNUC_HAS_ATTRIBUTE(always_inline, 4, 0, 0) ||                       \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0) ||                                    \
-    BROTLI_SUNPRO_VERSION_CHECK(5, 11, 0) ||                                   \
-    BROTLI_ARM_VERSION_CHECK(4, 1, 0) ||                                       \
-    BROTLI_IBM_VERSION_CHECK(10, 1, 0) ||                                      \
-    BROTLI_TI_VERSION_CHECK(8, 0, 0) ||                                        \
-    (BROTLI_TI_VERSION_CHECK(7, 3, 0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__))
-#define BROTLI_INLINE BROTLI_MAYBE_INLINE __attribute__((__always_inline__))
-#elif BROTLI_MSVC_VERSION_CHECK(12, 0, 0)
-#define BROTLI_INLINE BROTLI_MAYBE_INLINE __forceinline
-#elif BROTLI_TI_VERSION_CHECK(7, 0, 0) && defined(__cplusplus)
-#define BROTLI_INLINE BROTLI_MAYBE_INLINE _Pragma("FUNC_ALWAYS_INLINE;")
-#elif BROTLI_IAR_VERSION_CHECK(8, 0, 0)
-#define BROTLI_INLINE BROTLI_MAYBE_INLINE _Pragma("inline=forced")
-#else
-#define BROTLI_INLINE BROTLI_MAYBE_INLINE
-#endif
-
-#if BROTLI_GNUC_HAS_ATTRIBUTE(noinline, 4, 0, 0) ||                            \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0) ||                                    \
-    BROTLI_SUNPRO_VERSION_CHECK(5, 11, 0) ||                                   \
-    BROTLI_ARM_VERSION_CHECK(4, 1, 0) ||                                       \
-    BROTLI_IBM_VERSION_CHECK(10, 1, 0) ||                                      \
-    BROTLI_TI_VERSION_CHECK(8, 0, 0) ||                                        \
-    (BROTLI_TI_VERSION_CHECK(7, 3, 0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__))
-#define BROTLI_NOINLINE __attribute__((__noinline__))
-#elif BROTLI_MSVC_VERSION_CHECK(13, 10, 0)
-#define BROTLI_NOINLINE __declspec(noinline)
-#elif BROTLI_PGI_VERSION_CHECK(10, 2, 0)
-#define BROTLI_NOINLINE _Pragma("noinline")
-#elif BROTLI_TI_VERSION_CHECK(6, 0, 0) && defined(__cplusplus)
-#define BROTLI_NOINLINE _Pragma("FUNC_CANNOT_INLINE;")
-#elif BROTLI_IAR_VERSION_CHECK(8, 0, 0)
-#define BROTLI_NOINLINE _Pragma("inline=never")
-#else
-#define BROTLI_NOINLINE
-#endif
-
-/* BROTLI_INTERNAL could be defined to override visibility, e.g. for tests. */
-#if !defined(BROTLI_INTERNAL)
-#if defined(_WIN32) || defined(__CYGWIN__)
-#define BROTLI_INTERNAL
-#elif BROTLI_GNUC_VERSION_CHECK(3, 3, 0) ||                         \
-    BROTLI_TI_VERSION_CHECK(8, 0, 0) ||                             \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0) ||                         \
-    BROTLI_ARM_VERSION_CHECK(4, 1, 0) ||                            \
-    BROTLI_IBM_VERSION_CHECK(13, 1, 0) ||                           \
-    BROTLI_SUNPRO_VERSION_CHECK(5, 11, 0) ||                        \
-    (BROTLI_TI_VERSION_CHECK(7, 3, 0) &&                            \
-     defined(__TI_GNU_ATTRIBUTE_SUPPORT__) && defined(__TI_EABI__))
-#define BROTLI_INTERNAL __attribute__ ((visibility ("hidden")))
-#else
-#define BROTLI_INTERNAL
-#endif
-#endif
-
-/* <<< <<< <<< end of hedley macros. */
-
-#if BROTLI_GNUC_HAS_ATTRIBUTE(unused, 2, 7, 0) || \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0)
-#define BROTLI_UNUSED_FUNCTION static BROTLI_INLINE __attribute__ ((unused))
-#else
-#define BROTLI_UNUSED_FUNCTION static BROTLI_INLINE
-#endif
-
-#if (defined(__ARM_ARCH) && (__ARM_ARCH == 7)) || \
-    (defined(M_ARM) && (M_ARM == 7))
-#define BROTLI_TARGET_ARMV7
-#endif  /* ARMv7 */
-
-#if (defined(__ARM_ARCH) && (__ARM_ARCH == 8)) || \
-    defined(__aarch64__) || defined(__ARM64_ARCH_8__)
-#define BROTLI_TARGET_ARMV8_ANY
-
-#if defined(__ARM_32BIT_STATE)
-#define BROTLI_TARGET_ARMV8_32
-#elif defined(__ARM_64BIT_STATE)
-#define BROTLI_TARGET_ARMV8_64
-#endif
-
-#endif  /* ARMv8 */
-
-#if defined(__i386) || defined(_M_IX86)
-#define BROTLI_TARGET_X86
-#endif
-
-#if defined(__x86_64__) || defined(_M_X64)
-#define BROTLI_TARGET_X64
-#endif
-
-#if defined(__PPC64__)
-#define BROTLI_TARGET_POWERPC64
-#endif
-
-#if defined(__riscv) && defined(__riscv_xlen) && __riscv_xlen == 64
-#define BROTLI_TARGET_RISCV64
-#endif
-
-#if defined(BROTLI_BUILD_64_BIT)
-#define BROTLI_64_BITS 1
-#elif defined(BROTLI_BUILD_32_BIT)
-#define BROTLI_64_BITS 0
-#elif defined(BROTLI_TARGET_X64) || defined(BROTLI_TARGET_ARMV8_64) || \
-    defined(BROTLI_TARGET_POWERPC64) || defined(BROTLI_TARGET_RISCV64)
-#define BROTLI_64_BITS 1
-#else
-#define BROTLI_64_BITS 0
-#endif
-
-#if (BROTLI_64_BITS)
-#define brotli_reg_t uint64_t
-#else
-#define brotli_reg_t uint32_t
-#endif
-
-#if defined(BROTLI_BUILD_BIG_ENDIAN)
-#define BROTLI_BIG_ENDIAN 1
-#elif defined(BROTLI_BUILD_LITTLE_ENDIAN)
-#define BROTLI_LITTLE_ENDIAN 1
-#elif defined(BROTLI_BUILD_ENDIAN_NEUTRAL)
-/* Just break elif chain. */
-#elif defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__)
-#define BROTLI_LITTLE_ENDIAN 1
-#elif defined(_WIN32) || defined(BROTLI_TARGET_X64)
-/* Win32 & x64 can currently always be assumed to be little endian */
-#define BROTLI_LITTLE_ENDIAN 1
-#elif defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)
-#define BROTLI_BIG_ENDIAN 1
-#elif defined(BROTLI_X_BYTE_ORDER)
-#if BROTLI_X_BYTE_ORDER == BROTLI_X_LITTLE_ENDIAN
-#define BROTLI_LITTLE_ENDIAN 1
-#elif BROTLI_X_BYTE_ORDER == BROTLI_X_BIG_ENDIAN
-#define BROTLI_BIG_ENDIAN 1
-#endif
-#endif  /* BROTLI_X_BYTE_ORDER */
-
-#if !defined(BROTLI_LITTLE_ENDIAN)
-#define BROTLI_LITTLE_ENDIAN 0
-#endif
-
-#if !defined(BROTLI_BIG_ENDIAN)
-#define BROTLI_BIG_ENDIAN 0
-#endif
-
-#if defined(BROTLI_X_BYTE_ORDER)
-#undef BROTLI_X_BYTE_ORDER
-#undef BROTLI_X_LITTLE_ENDIAN
-#undef BROTLI_X_BIG_ENDIAN
-#endif
-
-#if defined(BROTLI_BUILD_PORTABLE)
-#define BROTLI_ALIGNED_READ (!!1)
-#elif defined(BROTLI_TARGET_X86) || defined(BROTLI_TARGET_X64) || \
-    defined(BROTLI_TARGET_ARMV7) || defined(BROTLI_TARGET_ARMV8_ANY) || \
-    defined(BROTLI_TARGET_RISCV64)
-/* Allow unaligned read only for white-listed CPUs. */
-#define BROTLI_ALIGNED_READ (!!0)
-#else
-#define BROTLI_ALIGNED_READ (!!1)
-#endif
-
-#if BROTLI_ALIGNED_READ
-/* Portable unaligned memory access: read / write values via memcpy. */
-static BROTLI_INLINE uint16_t BrotliUnalignedRead16(const void* p) {
-  uint16_t t;
-  memcpy(&t, p, sizeof t);
-  return t;
-}
-static BROTLI_INLINE uint32_t BrotliUnalignedRead32(const void* p) {
-  uint32_t t;
-  memcpy(&t, p, sizeof t);
-  return t;
-}
-static BROTLI_INLINE uint64_t BrotliUnalignedRead64(const void* p) {
-  uint64_t t;
-  memcpy(&t, p, sizeof t);
-  return t;
-}
-static BROTLI_INLINE void BrotliUnalignedWrite64(void* p, uint64_t v) {
-  memcpy(p, &v, sizeof v);
-}
-#else  /* BROTLI_ALIGNED_READ */
-/* Unaligned memory access is allowed: just cast pointer to requested type. */
-#if defined(ADDRESS_SANITIZER) || defined(THREAD_SANITIZER) || \
-    defined(MEMORY_SANITIZER)
-/* Consider we have an unaligned load/store of 4 bytes from address 0x...05.
-   AddressSanitizer will treat it as a 3-byte access to the range 05:07 and
-   will miss a bug if 08 is the first unaddressable byte.
-   ThreadSanitizer will also treat this as a 3-byte access to 05:07 and will
-   miss a race between this access and some other accesses to 08.
-   MemorySanitizer will correctly propagate the shadow on unaligned stores
-   and correctly report bugs on unaligned loads, but it may not properly
-   update and report the origin of the uninitialized memory.
-   For all three tools, replacing an unaligned access with a tool-specific
-   callback solves the problem. */
-#if defined(__cplusplus)
-extern "C" {
-#endif  /* __cplusplus */
-  uint16_t __sanitizer_unaligned_load16(const void* p);
-  uint32_t __sanitizer_unaligned_load32(const void* p);
-  uint64_t __sanitizer_unaligned_load64(const void* p);
-  void __sanitizer_unaligned_store64(void* p, uint64_t v);
-#if defined(__cplusplus)
-}  /* extern "C" */
-#endif  /* __cplusplus */
-#define BrotliUnalignedRead16 __sanitizer_unaligned_load16
-#define BrotliUnalignedRead32 __sanitizer_unaligned_load32
-#define BrotliUnalignedRead64 __sanitizer_unaligned_load64
-#define BrotliUnalignedWrite64 __sanitizer_unaligned_store64
-#else
-static BROTLI_INLINE uint16_t BrotliUnalignedRead16(const void* p) {
-  return *(const uint16_t*)p;
-}
-static BROTLI_INLINE uint32_t BrotliUnalignedRead32(const void* p) {
-  return *(const uint32_t*)p;
-}
-#if (BROTLI_64_BITS)
-static BROTLI_INLINE uint64_t BrotliUnalignedRead64(const void* p) {
-  return *(const uint64_t*)p;
-}
-static BROTLI_INLINE void BrotliUnalignedWrite64(void* p, uint64_t v) {
-  *(uint64_t*)p = v;
-}
-#else  /* BROTLI_64_BITS */
-/* Avoid emitting LDRD / STRD, which require properly aligned address. */
-/* If __attribute__(aligned) is available, use that. Otherwise, memcpy. */
-
-#if BROTLI_GNUC_HAS_ATTRIBUTE(aligned, 2, 7, 0)
-typedef  __attribute__((aligned(1))) uint64_t brotli_unaligned_uint64_t;
-
-static BROTLI_INLINE uint64_t BrotliUnalignedRead64(const void* p) {
-  return (uint64_t) ((brotli_unaligned_uint64_t*) p)[0];
-}
-static BROTLI_INLINE void BrotliUnalignedWrite64(void* p, uint64_t v) {
-  brotli_unaligned_uint64_t* dwords = (brotli_unaligned_uint64_t*) p;
-  dwords[0] = (brotli_unaligned_uint64_t) v;
-}
-#else /* BROTLI_GNUC_HAS_ATTRIBUTE(aligned, 2, 7, 0) */
-static BROTLI_INLINE uint64_t BrotliUnalignedRead64(const void* p) {
-  uint64_t v;
-  memcpy(&v, p, sizeof(uint64_t));
-  return v;
-}
-
-static BROTLI_INLINE void BrotliUnalignedWrite64(void* p, uint64_t v) {
-  memcpy(p, &v, sizeof(uint64_t));
-}
-#endif  /* BROTLI_GNUC_HAS_ATTRIBUTE(aligned, 2, 7, 0) */
-#endif  /* BROTLI_64_BITS */
-#endif  /* ASAN / TSAN / MSAN */
-#endif  /* BROTLI_ALIGNED_READ */
-
-#if BROTLI_LITTLE_ENDIAN
-/* Straight endianness. Just read / write values. */
-#define BROTLI_UNALIGNED_LOAD16LE BrotliUnalignedRead16
-#define BROTLI_UNALIGNED_LOAD32LE BrotliUnalignedRead32
-#define BROTLI_UNALIGNED_LOAD64LE BrotliUnalignedRead64
-#define BROTLI_UNALIGNED_STORE64LE BrotliUnalignedWrite64
-#elif BROTLI_BIG_ENDIAN  /* BROTLI_LITTLE_ENDIAN */
-/* Explain compiler to byte-swap values. */
-#define BROTLI_BSWAP16_(V) ((uint16_t)( \
-  (((V) & 0xFFU) << 8) | \
-  (((V) >> 8) & 0xFFU)))
-static BROTLI_INLINE uint16_t BROTLI_UNALIGNED_LOAD16LE(const void* p) {
-  uint16_t value = BrotliUnalignedRead16(p);
-  return BROTLI_BSWAP16_(value);
-}
-#define BROTLI_BSWAP32_(V) ( \
-  (((V) & 0xFFU) << 24) | (((V) & 0xFF00U) << 8) | \
-  (((V) >> 8) & 0xFF00U) | (((V) >> 24) & 0xFFU))
-static BROTLI_INLINE uint32_t BROTLI_UNALIGNED_LOAD32LE(const void* p) {
-  uint32_t value = BrotliUnalignedRead32(p);
-  return BROTLI_BSWAP32_(value);
-}
-#define BROTLI_BSWAP64_(V) ( \
-  (((V) & 0xFFU) << 56) | (((V) & 0xFF00U) << 40) | \
-  (((V) & 0xFF0000U) << 24) | (((V) & 0xFF000000U) << 8) | \
-  (((V) >> 8) & 0xFF000000U) | (((V) >> 24) & 0xFF0000U) | \
-  (((V) >> 40) & 0xFF00U) | (((V) >> 56) & 0xFFU))
-static BROTLI_INLINE uint64_t BROTLI_UNALIGNED_LOAD64LE(const void* p) {
-  uint64_t value = BrotliUnalignedRead64(p);
-  return BROTLI_BSWAP64_(value);
-}
-static BROTLI_INLINE void BROTLI_UNALIGNED_STORE64LE(void* p, uint64_t v) {
-  uint64_t value = BROTLI_BSWAP64_(v);
-  BrotliUnalignedWrite64(p, value);
-}
-#else  /* BROTLI_LITTLE_ENDIAN */
-/* Read / store values byte-wise; hopefully compiler will understand. */
-static BROTLI_INLINE uint16_t BROTLI_UNALIGNED_LOAD16LE(const void* p) {
-  const uint8_t* in = (const uint8_t*)p;
-  return (uint16_t)(in[0] | (in[1] << 8));
-}
-static BROTLI_INLINE uint32_t BROTLI_UNALIGNED_LOAD32LE(const void* p) {
-  const uint8_t* in = (const uint8_t*)p;
-  uint32_t value = (uint32_t)(in[0]);
-  value |= (uint32_t)(in[1]) << 8;
-  value |= (uint32_t)(in[2]) << 16;
-  value |= (uint32_t)(in[3]) << 24;
-  return value;
-}
-static BROTLI_INLINE uint64_t BROTLI_UNALIGNED_LOAD64LE(const void* p) {
-  const uint8_t* in = (const uint8_t*)p;
-  uint64_t value = (uint64_t)(in[0]);
-  value |= (uint64_t)(in[1]) << 8;
-  value |= (uint64_t)(in[2]) << 16;
-  value |= (uint64_t)(in[3]) << 24;
-  value |= (uint64_t)(in[4]) << 32;
-  value |= (uint64_t)(in[5]) << 40;
-  value |= (uint64_t)(in[6]) << 48;
-  value |= (uint64_t)(in[7]) << 56;
-  return value;
-}
-static BROTLI_INLINE void BROTLI_UNALIGNED_STORE64LE(void* p, uint64_t v) {
-  uint8_t* out = (uint8_t*)p;
-  out[0] = (uint8_t)v;
-  out[1] = (uint8_t)(v >> 8);
-  out[2] = (uint8_t)(v >> 16);
-  out[3] = (uint8_t)(v >> 24);
-  out[4] = (uint8_t)(v >> 32);
-  out[5] = (uint8_t)(v >> 40);
-  out[6] = (uint8_t)(v >> 48);
-  out[7] = (uint8_t)(v >> 56);
-}
-#endif  /* BROTLI_LITTLE_ENDIAN */
-
-/* BROTLI_IS_CONSTANT macros returns true for compile-time constants. */
-#if BROTLI_GNUC_HAS_BUILTIN(__builtin_constant_p, 3, 0, 1) || \
-    BROTLI_INTEL_VERSION_CHECK(16, 0, 0)
-#define BROTLI_IS_CONSTANT(x) (!!__builtin_constant_p(x))
-#else
-#define BROTLI_IS_CONSTANT(x) (!!0)
-#endif
-
-#if defined(BROTLI_TARGET_ARMV7) || defined(BROTLI_TARGET_ARMV8_ANY)
-#define BROTLI_HAS_UBFX (!!1)
-#else
-#define BROTLI_HAS_UBFX (!!0)
-#endif
-
-#if defined(BROTLI_ENABLE_LOG)
-#define BROTLI_DCHECK(x) assert(x)
-#define BROTLI_LOG(x) printf x
-#else
-#define BROTLI_DCHECK(x)
-#define BROTLI_LOG(x)
-#endif
-
-#if defined(BROTLI_DEBUG) || defined(BROTLI_ENABLE_LOG)
-static BROTLI_INLINE void BrotliDump(const char* f, int l, const char* fn) {
-  fprintf(stderr, "%s:%d (%s)\n", f, l, fn);
-  fflush(stderr);
-}
-#define BROTLI_DUMP() BrotliDump(__FILE__, __LINE__, __FUNCTION__)
-#else
-#define BROTLI_DUMP() (void)(0)
-#endif
-
-/* TODO: add appropriate icc/sunpro/arm/ibm/ti checks. */
-#if (BROTLI_GNUC_VERSION_CHECK(3, 0, 0) || defined(__llvm__)) && \
-    !defined(BROTLI_BUILD_NO_RBIT)
-#if defined(BROTLI_TARGET_ARMV7) || defined(BROTLI_TARGET_ARMV8_ANY)
-/* TODO: detect ARMv6T2 and enable this code for it. */
-static BROTLI_INLINE brotli_reg_t BrotliRBit(brotli_reg_t input) {
-  brotli_reg_t output;
-  __asm__("rbit %0, %1\n" : "=r"(output) : "r"(input));
-  return output;
-}
-#define BROTLI_RBIT(x) BrotliRBit(x)
-#endif  /* armv7 / armv8 */
-#endif  /* gcc || clang */
-#if !defined(BROTLI_RBIT)
-static BROTLI_INLINE void BrotliRBit(void) { /* Should break build if used. */ }
-#endif  /* BROTLI_RBIT */
-
-#define BROTLI_REPEAT(N, X) {     \
-  if ((N & 1) != 0) {X;}          \
-  if ((N & 2) != 0) {X; X;}       \
-  if ((N & 4) != 0) {X; X; X; X;} \
-}
-
-#define BROTLI_UNUSED(X) (void)(X)
-
-#define BROTLI_MIN_MAX(T)                                                      \
-  static BROTLI_INLINE T brotli_min_ ## T (T a, T b) { return a < b ? a : b; } \
-  static BROTLI_INLINE T brotli_max_ ## T (T a, T b) { return a > b ? a : b; }
-BROTLI_MIN_MAX(double) BROTLI_MIN_MAX(float) BROTLI_MIN_MAX(int)
-BROTLI_MIN_MAX(size_t) BROTLI_MIN_MAX(uint32_t) BROTLI_MIN_MAX(uint8_t)
-#undef BROTLI_MIN_MAX
-#define BROTLI_MIN(T, A, B) (brotli_min_ ## T((A), (B)))
-#define BROTLI_MAX(T, A, B) (brotli_max_ ## T((A), (B)))
-
-#define BROTLI_SWAP(T, A, I, J) { \
-  T __brotli_swap_tmp = (A)[(I)]; \
-  (A)[(I)] = (A)[(J)];            \
-  (A)[(J)] = __brotli_swap_tmp;   \
-}
-
-/* Default brotli_alloc_func */
-static void* BrotliDefaultAllocFunc(void* opaque, size_t size) {
-  BROTLI_UNUSED(opaque);
-  return malloc(size);
-}
-
-/* Default brotli_free_func */
-static void BrotliDefaultFreeFunc(void* opaque, void* address) {
-  BROTLI_UNUSED(opaque);
-  free(address);
-}
-
-BROTLI_UNUSED_FUNCTION void BrotliSuppressUnusedFunctions(void) {
-  BROTLI_UNUSED(&BrotliSuppressUnusedFunctions);
-  BROTLI_UNUSED(&BrotliUnalignedRead16);
-  BROTLI_UNUSED(&BrotliUnalignedRead32);
-  BROTLI_UNUSED(&BrotliUnalignedRead64);
-  BROTLI_UNUSED(&BrotliUnalignedWrite64);
-  BROTLI_UNUSED(&BROTLI_UNALIGNED_LOAD16LE);
-  BROTLI_UNUSED(&BROTLI_UNALIGNED_LOAD32LE);
-  BROTLI_UNUSED(&BROTLI_UNALIGNED_LOAD64LE);
-  BROTLI_UNUSED(&BROTLI_UNALIGNED_STORE64LE);
-  BROTLI_UNUSED(&BrotliRBit);
-  BROTLI_UNUSED(&brotli_min_double);
-  BROTLI_UNUSED(&brotli_max_double);
-  BROTLI_UNUSED(&brotli_min_float);
-  BROTLI_UNUSED(&brotli_max_float);
-  BROTLI_UNUSED(&brotli_min_int);
-  BROTLI_UNUSED(&brotli_max_int);
-  BROTLI_UNUSED(&brotli_min_size_t);
-  BROTLI_UNUSED(&brotli_max_size_t);
-  BROTLI_UNUSED(&brotli_min_uint32_t);
-  BROTLI_UNUSED(&brotli_max_uint32_t);
-  BROTLI_UNUSED(&brotli_min_uint8_t);
-  BROTLI_UNUSED(&brotli_max_uint8_t);
-  BROTLI_UNUSED(&BrotliDefaultAllocFunc);
-  BROTLI_UNUSED(&BrotliDefaultFreeFunc);
-#if defined(BROTLI_DEBUG) || defined(BROTLI_ENABLE_LOG)
-  BROTLI_UNUSED(&BrotliDump);
-#endif
-}
-
-#endif  /* BROTLI_COMMON_PLATFORM_H_ */

BaseTools/Source/C/BrotliCompress/common/transform.c

@@ -1,235 +0,0 @@
-/* Copyright 2013 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-#include "./transform.h"
-
-#if defined(__cplusplus) || defined(c_plusplus)
-extern "C" {
-#endif
-
-/* RFC 7932 transforms string data */
-static const char kPrefixSuffix[217] =
-      "\1 \2, \10 of the \4 of \2s \1.\5 and \4 "
-/* 0x  _0 _2  __5        _E    _3  _6 _8     _E */
-      "in \1\"\4 to \2\">\1\n\2. \1]\5 for \3 a \6 "
-/* 2x     _3_ _5    _A_  _D_ _F  _2 _4     _A   _E */
-      "that \1\'\6 with \6 from \4 by \1(\6. T"
-/* 4x       _5_ _7      _E      _5    _A _C */
-      "he \4 on \4 as \4 is \4ing \2\n\t\1:\3ed "
-/* 6x     _3    _8    _D    _2    _7_ _ _A _C */
-      "\2=\"\4 at \3ly \1,\2=\'\5.com/\7. This \5"
-/* 8x  _0 _ _3    _8   _C _E _ _1     _7       _F */
-      " not \3er \3al \4ful \4ive \5less \4es"
-/* Ax       _5   _9   _D    _2    _7     _D */
-      "t \4ize \2\xc2\xa0\4ous \5 the \2e \0";
-/* Cx    _2    _7___ ___ _A    _F     _5  _8 */
-
-static const uint16_t kPrefixSuffixMap[50] = {
-  0x00, 0x02, 0x05, 0x0E, 0x13, 0x16, 0x18, 0x1E, 0x23, 0x25,
-  0x2A, 0x2D, 0x2F, 0x32, 0x34, 0x3A, 0x3E, 0x45, 0x47, 0x4E,
-  0x55, 0x5A, 0x5C, 0x63, 0x68, 0x6D, 0x72, 0x77, 0x7A, 0x7C,
-  0x80, 0x83, 0x88, 0x8C, 0x8E, 0x91, 0x97, 0x9F, 0xA5, 0xA9,
-  0xAD, 0xB2, 0xB7, 0xBD, 0xC2, 0xC7, 0xCA, 0xCF, 0xD5, 0xD8
-};
-
-/* RFC 7932 transforms */
-static const uint8_t kTransformsData[] = {
-  49, BROTLI_TRANSFORM_IDENTITY, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 0,
-   0, BROTLI_TRANSFORM_IDENTITY, 0,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_1, 49,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 0,
-  49, BROTLI_TRANSFORM_IDENTITY, 47,
-   0, BROTLI_TRANSFORM_IDENTITY, 49,
-   4, BROTLI_TRANSFORM_IDENTITY, 0,
-  49, BROTLI_TRANSFORM_IDENTITY, 3,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 6,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_2, 49,
-  49, BROTLI_TRANSFORM_OMIT_LAST_1, 49,
-   1, BROTLI_TRANSFORM_IDENTITY, 0,
-  49, BROTLI_TRANSFORM_IDENTITY, 1,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 0,
-  49, BROTLI_TRANSFORM_IDENTITY, 7,
-  49, BROTLI_TRANSFORM_IDENTITY, 9,
-  48, BROTLI_TRANSFORM_IDENTITY, 0,
-  49, BROTLI_TRANSFORM_IDENTITY, 8,
-  49, BROTLI_TRANSFORM_IDENTITY, 5,
-  49, BROTLI_TRANSFORM_IDENTITY, 10,
-  49, BROTLI_TRANSFORM_IDENTITY, 11,
-  49, BROTLI_TRANSFORM_OMIT_LAST_3, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 13,
-  49, BROTLI_TRANSFORM_IDENTITY, 14,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_3, 49,
-  49, BROTLI_TRANSFORM_OMIT_LAST_2, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 15,
-  49, BROTLI_TRANSFORM_IDENTITY, 16,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 12,
-   5, BROTLI_TRANSFORM_IDENTITY, 49,
-   0, BROTLI_TRANSFORM_IDENTITY, 1,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_4, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 18,
-  49, BROTLI_TRANSFORM_IDENTITY, 17,
-  49, BROTLI_TRANSFORM_IDENTITY, 19,
-  49, BROTLI_TRANSFORM_IDENTITY, 20,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_5, 49,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_6, 49,
-  47, BROTLI_TRANSFORM_IDENTITY, 49,
-  49, BROTLI_TRANSFORM_OMIT_LAST_4, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 22,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 23,
-  49, BROTLI_TRANSFORM_IDENTITY, 24,
-  49, BROTLI_TRANSFORM_IDENTITY, 25,
-  49, BROTLI_TRANSFORM_OMIT_LAST_7, 49,
-  49, BROTLI_TRANSFORM_OMIT_LAST_1, 26,
-  49, BROTLI_TRANSFORM_IDENTITY, 27,
-  49, BROTLI_TRANSFORM_IDENTITY, 28,
-   0, BROTLI_TRANSFORM_IDENTITY, 12,
-  49, BROTLI_TRANSFORM_IDENTITY, 29,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_9, 49,
-  49, BROTLI_TRANSFORM_OMIT_FIRST_7, 49,
-  49, BROTLI_TRANSFORM_OMIT_LAST_6, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 21,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 1,
-  49, BROTLI_TRANSFORM_OMIT_LAST_8, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 31,
-  49, BROTLI_TRANSFORM_IDENTITY, 32,
-  47, BROTLI_TRANSFORM_IDENTITY, 3,
-  49, BROTLI_TRANSFORM_OMIT_LAST_5, 49,
-  49, BROTLI_TRANSFORM_OMIT_LAST_9, 49,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 1,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 8,
-   5, BROTLI_TRANSFORM_IDENTITY, 21,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 0,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 10,
-  49, BROTLI_TRANSFORM_IDENTITY, 30,
-   0, BROTLI_TRANSFORM_IDENTITY, 5,
-  35, BROTLI_TRANSFORM_IDENTITY, 49,
-  47, BROTLI_TRANSFORM_IDENTITY, 2,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 17,
-  49, BROTLI_TRANSFORM_IDENTITY, 36,
-  49, BROTLI_TRANSFORM_IDENTITY, 33,
-   5, BROTLI_TRANSFORM_IDENTITY, 0,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 21,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 5,
-  49, BROTLI_TRANSFORM_IDENTITY, 37,
-   0, BROTLI_TRANSFORM_IDENTITY, 30,
-  49, BROTLI_TRANSFORM_IDENTITY, 38,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 0,
-  49, BROTLI_TRANSFORM_IDENTITY, 39,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 49,
-  49, BROTLI_TRANSFORM_IDENTITY, 34,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 8,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 12,
-   0, BROTLI_TRANSFORM_IDENTITY, 21,
-  49, BROTLI_TRANSFORM_IDENTITY, 40,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 12,
-  49, BROTLI_TRANSFORM_IDENTITY, 41,
-  49, BROTLI_TRANSFORM_IDENTITY, 42,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 17,
-  49, BROTLI_TRANSFORM_IDENTITY, 43,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 5,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 10,
-   0, BROTLI_TRANSFORM_IDENTITY, 34,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 33,
-  49, BROTLI_TRANSFORM_IDENTITY, 44,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 5,
-  45, BROTLI_TRANSFORM_IDENTITY, 49,
-   0, BROTLI_TRANSFORM_IDENTITY, 33,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 30,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 30,
-  49, BROTLI_TRANSFORM_IDENTITY, 46,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 1,
-  49, BROTLI_TRANSFORM_UPPERCASE_FIRST, 34,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 33,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 30,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 1,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 33,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 21,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 12,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 5,
-  49, BROTLI_TRANSFORM_UPPERCASE_ALL, 34,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 12,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 30,
-   0, BROTLI_TRANSFORM_UPPERCASE_ALL, 34,
-   0, BROTLI_TRANSFORM_UPPERCASE_FIRST, 34,
-};
-
-static BrotliTransforms kBrotliTransforms = {
-  sizeof(kPrefixSuffix),
-  (const uint8_t*)kPrefixSuffix,
-  kPrefixSuffixMap,
-  sizeof(kTransformsData) / (3 * sizeof(kTransformsData[0])),
-  kTransformsData,
-  {0, 12, 27, 23, 42, 63, 56, 48, 59, 64}
-};
-
-const BrotliTransforms* BrotliGetTransforms(void) {
-  return &kBrotliTransforms;
-}
-
-static int ToUpperCase(uint8_t* p) {
-  if (p[0] < 0xC0) {
-    if (p[0] >= 'a' && p[0] <= 'z') {
-      p[0] ^= 32;
-    }
-    return 1;
-  }
-  /* An overly simplified uppercasing model for UTF-8. */
-  if (p[0] < 0xE0) {
-    p[1] ^= 32;
-    return 2;
-  }
-  /* An arbitrary transform for three byte characters. */
-  p[2] ^= 5;
-  return 3;
-}
-
-int BrotliTransformDictionaryWord(uint8_t* dst, const uint8_t* word, int len,
-    const BrotliTransforms* transforms, int transfom_idx) {
-  int idx = 0;
-  const uint8_t* prefix = BROTLI_TRANSFORM_PREFIX(transforms, transfom_idx);
-  uint8_t type = BROTLI_TRANSFORM_TYPE(transforms, transfom_idx);
-  const uint8_t* suffix = BROTLI_TRANSFORM_SUFFIX(transforms, transfom_idx);
-  {
-    int prefix_len = *prefix++;
-    while (prefix_len--) { dst[idx++] = *prefix++; }
-  }
-  {
-    const int t = type;
-    int i = 0;
-    if (t <= BROTLI_TRANSFORM_OMIT_LAST_9) {
-      len -= t;
-    } else if (t >= BROTLI_TRANSFORM_OMIT_FIRST_1
-        && t <= BROTLI_TRANSFORM_OMIT_FIRST_9) {
-      int skip = t - (BROTLI_TRANSFORM_OMIT_FIRST_1 - 1);
-      word += skip;
-      len -= skip;
-    }
-    while (i < len) { dst[idx++] = word[i++]; }
-    if (t == BROTLI_TRANSFORM_UPPERCASE_FIRST) {
-      ToUpperCase(&dst[idx - len]);
-    } else if (t == BROTLI_TRANSFORM_UPPERCASE_ALL) {
-      uint8_t* uppercase = &dst[idx - len];
-      while (len > 0) {
-        int step = ToUpperCase(uppercase);
-        uppercase += step;
-        len -= step;
-      }
-    }
-  }
-  {
-    int suffix_len = *suffix++;
-    while (suffix_len--) { dst[idx++] = *suffix++; }
-    return idx;
-  }
-}
-
-#if defined(__cplusplus) || defined(c_plusplus)
-}  /* extern "C" */
-#endif

BaseTools/Source/C/BrotliCompress/common/transform.h

@@ -1,80 +0,0 @@
-/* transforms is a part of ABI, but not API.
-
-   It means that there are some functions that are supposed to be in "common"
-   library, but header itself is not placed into include/brotli. This way,
-   aforementioned functions will be available only to brotli internals.
- */
-
-#ifndef BROTLI_COMMON_TRANSFORM_H_
-#define BROTLI_COMMON_TRANSFORM_H_
-
-#include <brotli/port.h>
-#include <brotli/types.h>
-
-#if defined(__cplusplus) || defined(c_plusplus)
-extern "C" {
-#endif
-
-enum BrotliWordTransformType {
-  BROTLI_TRANSFORM_IDENTITY = 0,
-  BROTLI_TRANSFORM_OMIT_LAST_1 = 1,
-  BROTLI_TRANSFORM_OMIT_LAST_2 = 2,
-  BROTLI_TRANSFORM_OMIT_LAST_3 = 3,
-  BROTLI_TRANSFORM_OMIT_LAST_4 = 4,
-  BROTLI_TRANSFORM_OMIT_LAST_5 = 5,
-  BROTLI_TRANSFORM_OMIT_LAST_6 = 6,
-  BROTLI_TRANSFORM_OMIT_LAST_7 = 7,
-  BROTLI_TRANSFORM_OMIT_LAST_8 = 8,
-  BROTLI_TRANSFORM_OMIT_LAST_9 = 9,
-  BROTLI_TRANSFORM_UPPERCASE_FIRST = 10,
-  BROTLI_TRANSFORM_UPPERCASE_ALL = 11,
-  BROTLI_TRANSFORM_OMIT_FIRST_1 = 12,
-  BROTLI_TRANSFORM_OMIT_FIRST_2 = 13,
-  BROTLI_TRANSFORM_OMIT_FIRST_3 = 14,
-  BROTLI_TRANSFORM_OMIT_FIRST_4 = 15,
-  BROTLI_TRANSFORM_OMIT_FIRST_5 = 16,
-  BROTLI_TRANSFORM_OMIT_FIRST_6 = 17,
-  BROTLI_TRANSFORM_OMIT_FIRST_7 = 18,
-  BROTLI_TRANSFORM_OMIT_FIRST_8 = 19,
-  BROTLI_TRANSFORM_OMIT_FIRST_9 = 20,
-  BROTLI_NUM_TRANSFORM_TYPES  /* Counts transforms, not a transform itself. */
-};
-
-#define BROTLI_TRANSFORMS_MAX_CUT_OFF BROTLI_TRANSFORM_OMIT_LAST_9
-
-typedef struct BrotliTransforms {
-  uint16_t prefix_suffix_size;
-  /* Last character must be null, so prefix_suffix_size must be at least 1. */
-  const uint8_t* prefix_suffix;
-  const uint16_t* prefix_suffix_map;
-  uint32_t num_transforms;
-  /* Each entry is a [prefix_id, transform, suffix_id] triplet. */
-  const uint8_t* transforms;
-  /* Indices of transforms like ["", BROTLI_TRANSFORM_OMIT_LAST_#, ""].
-     0-th element corresponds to ["", BROTLI_TRANSFORM_IDENTITY, ""].
-     -1, if cut-off transform does not exist. */
-  int16_t cutOffTransforms[BROTLI_TRANSFORMS_MAX_CUT_OFF + 1];
-} BrotliTransforms;
-
-/* T is BrotliTransforms*; result is uint8_t. */
-#define BROTLI_TRANSFORM_PREFIX_ID(T, I) ((T)->transforms[((I) * 3) + 0])
-#define BROTLI_TRANSFORM_TYPE(T, I)      ((T)->transforms[((I) * 3) + 1])
-#define BROTLI_TRANSFORM_SUFFIX_ID(T, I) ((T)->transforms[((I) * 3) + 2])
-
-/* T is BrotliTransforms*; result is const uint8_t*. */
-#define BROTLI_TRANSFORM_PREFIX(T, I) (&(T)->prefix_suffix[ \
-    (T)->prefix_suffix_map[BROTLI_TRANSFORM_PREFIX_ID(T, I)]])
-#define BROTLI_TRANSFORM_SUFFIX(T, I) (&(T)->prefix_suffix[ \
-    (T)->prefix_suffix_map[BROTLI_TRANSFORM_SUFFIX_ID(T, I)]])
-
-BROTLI_COMMON_API const BrotliTransforms* BrotliGetTransforms(void);
-
-BROTLI_COMMON_API int BrotliTransformDictionaryWord(
-    uint8_t* dst, const uint8_t* word, int len,
-    const BrotliTransforms* transforms, int transform_idx);
-
-#if defined(__cplusplus) || defined(c_plusplus)
-}  /* extern "C" */
-#endif
-
-#endif  /* BROTLI_COMMON_TRANSFORM_H_ */

BaseTools/Source/C/BrotliCompress/common/version.h

@@ -1,26 +0,0 @@
-/* Copyright 2016 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Version definition. */
-
-#ifndef BROTLI_COMMON_VERSION_H_
-#define BROTLI_COMMON_VERSION_H_
-
-/* This macro should only be used when library is compiled together with client.
-   If library is dynamically linked, use BrotliDecoderVersion and
-   BrotliEncoderVersion methods. */
-
-/* Semantic version, calculated as (MAJOR << 24) | (MINOR << 12) | PATCH */
-#define BROTLI_VERSION 0x1000006
-
-/* This macro is used by build system to produce Libtool-friendly soname. See
-   https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html
- */
-
-/* ABI version, calculated as (CURRENT << 24) | (REVISION << 12) | AGE */
-#define BROTLI_ABI_VERSION 0x1006000
-
-#endif  /* BROTLI_COMMON_VERSION_H_ */

BaseTools/Source/C/BrotliCompress/dec/bit_reader.c

@@ -1,48 +0,0 @@
-/* Copyright 2013 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Bit reading helpers */
-
-#include "./bit_reader.h"
-
-#include "../common/platform.h"
-#include <brotli/types.h>
-
-#if defined(__cplusplus) || defined(c_plusplus)
-extern "C" {
-#endif
-
-void BrotliInitBitReader(BrotliBitReader* const br) {
-  br->val_ = 0;
-  br->bit_pos_ = sizeof(br->val_) << 3;
-}
-
-BROTLI_BOOL BrotliWarmupBitReader(BrotliBitReader* const br) {
-  size_t aligned_read_mask = (sizeof(br->val_) >> 1) - 1;
-  /* Fixing alignment after unaligned BrotliFillWindow would result accumulator
-     overflow. If unalignment is caused by BrotliSafeReadBits, then there is
-     enough space in accumulator to fix alignment. */
-  if (!BROTLI_ALIGNED_READ) {
-    aligned_read_mask = 0;
-  }
-  if (BrotliGetAvailableBits(br) == 0) {
-    if (!BrotliPullByte(br)) {
-      return BROTLI_FALSE;
-    }
-  }
-
-  while ((((size_t)br->next_in) & aligned_read_mask) != 0) {
-    if (!BrotliPullByte(br)) {
-      /* If we consumed all the input, we don't care about the alignment. */
-      return BROTLI_TRUE;
-    }
-  }
-  return BROTLI_TRUE;
-}
-
-#if defined(__cplusplus) || defined(c_plusplus)
-}  /* extern "C" */
-#endif

BaseTools/Source/C/BrotliCompress/dec/bit_reader.h

@@ -1,309 +0,0 @@
-/* Copyright 2013 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Bit reading helpers */
-
-#ifndef BROTLI_DEC_BIT_READER_H_
-#define BROTLI_DEC_BIT_READER_H_
-
-#include <string.h>  /* memcpy */
-
-#include "../common/platform.h"
-#include <brotli/types.h>
-
-#if defined(__cplusplus) || defined(c_plusplus)
-extern "C" {
-#endif
-
-#define BROTLI_SHORT_FILL_BIT_WINDOW_READ (sizeof(brotli_reg_t) >> 1)
-
-static const uint32_t kBitMask[33] = {  0x00000000,
-    0x00000001, 0x00000003, 0x00000007, 0x0000000F,
-    0x0000001F, 0x0000003F, 0x0000007F, 0x000000FF,
-    0x000001FF, 0x000003FF, 0x000007FF, 0x00000FFF,
-    0x00001FFF, 0x00003FFF, 0x00007FFF, 0x0000FFFF,
-    0x0001FFFF, 0x0003FFFF, 0x0007FFFF, 0x000FFFFF,
-    0x001FFFFF, 0x003FFFFF, 0x007FFFFF, 0x00FFFFFF,
-    0x01FFFFFF, 0x03FFFFFF, 0x07FFFFFF, 0x0FFFFFFF,
-    0x1FFFFFFF, 0x3FFFFFFF, 0x7FFFFFFF, 0xFFFFFFFF
-};
-
-static BROTLI_INLINE uint32_t BitMask(uint32_t n) {
-  if (BROTLI_IS_CONSTANT(n) || BROTLI_HAS_UBFX) {
-    /* Masking with this expression turns to a single
-       "Unsigned Bit Field Extract" UBFX instruction on ARM. */
-    return ~((0xFFFFFFFFu) << n);
-  } else {
-    return kBitMask[n];
-  }
-}
-
-typedef struct {
-  brotli_reg_t val_;       /* pre-fetched bits */
-  uint32_t bit_pos_;       /* current bit-reading position in val_ */
-  const uint8_t* next_in;  /* the byte we're reading from */
-  size_t avail_in;
-} BrotliBitReader;
-
-typedef struct {
-  brotli_reg_t val_;
-  uint32_t bit_pos_;
-  const uint8_t* next_in;
-  size_t avail_in;
-} BrotliBitReaderState;
-
-/* Initializes the BrotliBitReader fields. */
-BROTLI_INTERNAL void BrotliInitBitReader(BrotliBitReader* const br);
-
-/* Ensures that accumulator is not empty.
-   May consume up to sizeof(brotli_reg_t) - 1 bytes of input.
-   Returns BROTLI_FALSE if data is required but there is no input available.
-   For BROTLI_ALIGNED_READ this function also prepares bit reader for aligned
-   reading. */
-BROTLI_INTERNAL BROTLI_BOOL BrotliWarmupBitReader(BrotliBitReader* const br);
-
-static BROTLI_INLINE void BrotliBitReaderSaveState(
-    BrotliBitReader* const from, BrotliBitReaderState* to) {
-  to->val_ = from->val_;
-  to->bit_pos_ = from->bit_pos_;
-  to->next_in = from->next_in;
-  to->avail_in = from->avail_in;
-}
-
-static BROTLI_INLINE void BrotliBitReaderRestoreState(
-    BrotliBitReader* const to, BrotliBitReaderState* from) {
-  to->val_ = from->val_;
-  to->bit_pos_ = from->bit_pos_;
-  to->next_in = from->next_in;
-  to->avail_in = from->avail_in;
-}
-
-static BROTLI_INLINE uint32_t BrotliGetAvailableBits(
-    const BrotliBitReader* br) {
-  return (BROTLI_64_BITS ? 64 : 32) - br->bit_pos_;
-}
-
-/* Returns amount of unread bytes the bit reader still has buffered from the
-   BrotliInput, including whole bytes in br->val_. */
-static BROTLI_INLINE size_t BrotliGetRemainingBytes(BrotliBitReader* br) {
-  return br->avail_in + (BrotliGetAvailableBits(br) >> 3);
-}
-
-/* Checks if there is at least |num| bytes left in the input ring-buffer
-   (excluding the bits remaining in br->val_). */
-static BROTLI_INLINE BROTLI_BOOL BrotliCheckInputAmount(
-    BrotliBitReader* const br, size_t num) {
-  return TO_BROTLI_BOOL(br->avail_in >= num);
-}
-
-/* Guarantees that there are at least |n_bits| + 1 bits in accumulator.
-   Precondition: accumulator contains at least 1 bit.
-   |n_bits| should be in the range [1..24] for regular build. For portable
-   non-64-bit little-endian build only 16 bits are safe to request. */
-static BROTLI_INLINE void BrotliFillBitWindow(
-    BrotliBitReader* const br, uint32_t n_bits) {
-#if (BROTLI_64_BITS)
-  if (!BROTLI_ALIGNED_READ && BROTLI_IS_CONSTANT(n_bits) && (n_bits <= 8)) {
-    if (br->bit_pos_ >= 56) {
-      br->val_ >>= 56;
-      br->bit_pos_ ^= 56;  /* here same as -= 56 because of the if condition */
-      br->val_ |= BROTLI_UNALIGNED_LOAD64LE(br->next_in) << 8;
-      br->avail_in -= 7;
-      br->next_in += 7;
-    }
-  } else if (
-      !BROTLI_ALIGNED_READ && BROTLI_IS_CONSTANT(n_bits) && (n_bits <= 16)) {
-    if (br->bit_pos_ >= 48) {
-      br->val_ >>= 48;
-      br->bit_pos_ ^= 48;  /* here same as -= 48 because of the if condition */
-      br->val_ |= BROTLI_UNALIGNED_LOAD64LE(br->next_in) << 16;
-      br->avail_in -= 6;
-      br->next_in += 6;
-    }
-  } else {
-    if (br->bit_pos_ >= 32) {
-      br->val_ >>= 32;
-      br->bit_pos_ ^= 32;  /* here same as -= 32 because of the if condition */
-      br->val_ |= ((uint64_t)BROTLI_UNALIGNED_LOAD32LE(br->next_in)) << 32;
-      br->avail_in -= BROTLI_SHORT_FILL_BIT_WINDOW_READ;
-      br->next_in += BROTLI_SHORT_FILL_BIT_WINDOW_READ;
-    }
-  }
-#else
-  if (!BROTLI_ALIGNED_READ && BROTLI_IS_CONSTANT(n_bits) && (n_bits <= 8)) {
-    if (br->bit_pos_ >= 24) {
-      br->val_ >>= 24;
-      br->bit_pos_ ^= 24;  /* here same as -= 24 because of the if condition */
-      br->val_ |= BROTLI_UNALIGNED_LOAD32LE(br->next_in) << 8;
-      br->avail_in -= 3;
-      br->next_in += 3;
-    }
-  } else {
-    if (br->bit_pos_ >= 16) {
-      br->val_ >>= 16;
-      br->bit_pos_ ^= 16;  /* here same as -= 16 because of the if condition */
-      br->val_ |= ((uint32_t)BROTLI_UNALIGNED_LOAD16LE(br->next_in)) << 16;
-      br->avail_in -= BROTLI_SHORT_FILL_BIT_WINDOW_READ;
-      br->next_in += BROTLI_SHORT_FILL_BIT_WINDOW_READ;
-    }
-  }
-#endif
-}
-
-/* Mostly like BrotliFillBitWindow, but guarantees only 16 bits and reads no
-   more than BROTLI_SHORT_FILL_BIT_WINDOW_READ bytes of input. */
-static BROTLI_INLINE void BrotliFillBitWindow16(BrotliBitReader* const br) {
-  BrotliFillBitWindow(br, 17);
-}
-
-/* Tries to pull one byte of input to accumulator.
-   Returns BROTLI_FALSE if there is no input available. */
-static BROTLI_INLINE BROTLI_BOOL BrotliPullByte(BrotliBitReader* const br) {
-  if (br->avail_in == 0) {
-    return BROTLI_FALSE;
-  }
-  br->val_ >>= 8;
-#if (BROTLI_64_BITS)
-  br->val_ |= ((uint64_t)*br->next_in) << 56;
-#else
-  br->val_ |= ((uint32_t)*br->next_in) << 24;
-#endif
-  br->bit_pos_ -= 8;
-  --br->avail_in;
-  ++br->next_in;
-  return BROTLI_TRUE;
-}
-
-/* Returns currently available bits.
-   The number of valid bits could be calculated by BrotliGetAvailableBits. */
-static BROTLI_INLINE brotli_reg_t BrotliGetBitsUnmasked(
-    BrotliBitReader* const br) {
-  return br->val_ >> br->bit_pos_;
-}
-
-/* Like BrotliGetBits, but does not mask the result.
-   The result contains at least 16 valid bits. */
-static BROTLI_INLINE uint32_t BrotliGet16BitsUnmasked(
-    BrotliBitReader* const br) {
-  BrotliFillBitWindow(br, 16);
-  return (uint32_t)BrotliGetBitsUnmasked(br);
-}
-
-/* Returns the specified number of bits from |br| without advancing bit
-   position. */
-static BROTLI_INLINE uint32_t BrotliGetBits(
-    BrotliBitReader* const br, uint32_t n_bits) {
-  BrotliFillBitWindow(br, n_bits);
-  return (uint32_t)BrotliGetBitsUnmasked(br) & BitMask(n_bits);
-}
-
-/* Tries to peek the specified amount of bits. Returns BROTLI_FALSE, if there
-   is not enough input. */
-static BROTLI_INLINE BROTLI_BOOL BrotliSafeGetBits(
-    BrotliBitReader* const br, uint32_t n_bits, uint32_t* val) {
-  while (BrotliGetAvailableBits(br) < n_bits) {
-    if (!BrotliPullByte(br)) {
-      return BROTLI_FALSE;
-    }
-  }
-  *val = (uint32_t)BrotliGetBitsUnmasked(br) & BitMask(n_bits);
-  return BROTLI_TRUE;
-}
-
-/* Advances the bit pos by |n_bits|. */
-static BROTLI_INLINE void BrotliDropBits(
-    BrotliBitReader* const br, uint32_t n_bits) {
-  br->bit_pos_ += n_bits;
-}
-
-static BROTLI_INLINE void BrotliBitReaderUnload(BrotliBitReader* br) {
-  uint32_t unused_bytes = BrotliGetAvailableBits(br) >> 3;
-  uint32_t unused_bits = unused_bytes << 3;
-  br->avail_in += unused_bytes;
-  br->next_in -= unused_bytes;
-  if (unused_bits == sizeof(br->val_) << 3) {
-    br->val_ = 0;
-  } else {
-    br->val_ <<= unused_bits;
-  }
-  br->bit_pos_ += unused_bits;
-}
-
-/* Reads the specified number of bits from |br| and advances the bit pos.
-   Precondition: accumulator MUST contain at least |n_bits|. */
-static BROTLI_INLINE void BrotliTakeBits(
-  BrotliBitReader* const br, uint32_t n_bits, uint32_t* val) {
-  *val = (uint32_t)BrotliGetBitsUnmasked(br) & BitMask(n_bits);
-  BROTLI_LOG(("[BrotliReadBits]  %d %d %d val: %6x\n",
-      (int)br->avail_in, (int)br->bit_pos_, (int)n_bits, (int)*val));
-  BrotliDropBits(br, n_bits);
-}
-
-/* Reads the specified number of bits from |br| and advances the bit pos.
-   Assumes that there is enough input to perform BrotliFillBitWindow. */
-static BROTLI_INLINE uint32_t BrotliReadBits(
-    BrotliBitReader* const br, uint32_t n_bits) {
-  if (BROTLI_64_BITS || (n_bits <= 16)) {
-    uint32_t val;
-    BrotliFillBitWindow(br, n_bits);
-    BrotliTakeBits(br, n_bits, &val);
-    return val;
-  } else {
-    uint32_t low_val;
-    uint32_t high_val;
-    BrotliFillBitWindow(br, 16);
-    BrotliTakeBits(br, 16, &low_val);
-    BrotliFillBitWindow(br, 8);
-    BrotliTakeBits(br, n_bits - 16, &high_val);
-    return low_val | (high_val << 16);
-  }
-}
-
-/* Tries to read the specified amount of bits. Returns BROTLI_FALSE, if there
-   is not enough input. |n_bits| MUST be positive. */
-static BROTLI_INLINE BROTLI_BOOL BrotliSafeReadBits(
-    BrotliBitReader* const br, uint32_t n_bits, uint32_t* val) {
-  while (BrotliGetAvailableBits(br) < n_bits) {
-    if (!BrotliPullByte(br)) {
-      return BROTLI_FALSE;
-    }
-  }
-  BrotliTakeBits(br, n_bits, val);
-  return BROTLI_TRUE;
-}
-
-/* Advances the bit reader position to the next byte boundary and verifies
-   that any skipped bits are set to zero. */
-static BROTLI_INLINE BROTLI_BOOL BrotliJumpToByteBoundary(BrotliBitReader* br) {
-  uint32_t pad_bits_count = BrotliGetAvailableBits(br) & 0x7;
-  uint32_t pad_bits = 0;
-  if (pad_bits_count != 0) {
-    BrotliTakeBits(br, pad_bits_count, &pad_bits);
-  }
-  return TO_BROTLI_BOOL(pad_bits == 0);
-}
-
-/* Copies remaining input bytes stored in the bit reader to the output. Value
-   |num| may not be larger than BrotliGetRemainingBytes. The bit reader must be
-   warmed up again after this. */
-static BROTLI_INLINE void BrotliCopyBytes(uint8_t* dest,
-                                          BrotliBitReader* br, size_t num) {
-  while (BrotliGetAvailableBits(br) >= 8 && num > 0) {
-    *dest = (uint8_t)BrotliGetBitsUnmasked(br);
-    BrotliDropBits(br, 8);
-    ++dest;
-    --num;
-  }
-  memcpy(dest, br->next_in, num);
-  br->avail_in -= num;
-  br->next_in += num;
-}
-
-#if defined(__cplusplus) || defined(c_plusplus)
-}  /* extern "C" */
-#endif
-
-#endif  /* BROTLI_DEC_BIT_READER_H_ */

BaseTools/Source/C/BrotliCompress/dec/huffman.c

@@ -1,356 +0,0 @@
-/* Copyright 2013 Google Inc. All Rights Reserved.
-
-   Distributed under MIT license.
-   See file LICENSE for detail or copy at https://opensource.org/licenses/MIT
-*/
-
-/* Utilities for building Huffman decoding tables. */
-
-#include "./huffman.h"
-
-#include <string.h>  /* memcpy, memset */
-
-#include "../common/constants.h"
-#include "../common/platform.h"
-#include <brotli/types.h>
-
-#if defined(__cplusplus) || defined(c_plusplus)
-extern "C" {
-#endif
-
-#define BROTLI_REVERSE_BITS_MAX 8
-
-#if defined(BROTLI_RBIT)
-#define BROTLI_REVERSE_BITS_BASE \
-  ((sizeof(brotli_reg_t) << 3) - BROTLI_REVERSE_BITS_MAX)
-#else
-#define BROTLI_REVERSE_BITS_BASE 0
-static uint8_t kReverseBits[1 << BROTLI_REVERSE_BITS_MAX] = {
-  0x00, 0x80, 0x40, 0xC0, 0x20, 0xA0, 0x60, 0xE0,
-  0x10, 0x90, 0x50, 0xD0, 0x30, 0xB0, 0x70, 0xF0,
-  0x08, 0x88, 0x48, 0xC8, 0x28, 0xA8, 0x68, 0xE8,
-  0x18, 0x98, 0x58, 0xD8, 0x38, 0xB8, 0x78, 0xF8,
-  0x04, 0x84, 0x44, 0xC4, 0x24, 0xA4, 0x64, 0xE4,
-  0x14, 0x94, 0x54, 0xD4, 0x34, 0xB4, 0x74, 0xF4,
-  0x0C, 0x8C, 0x4C, 0xCC, 0x2C, 0xAC, 0x6C, 0xEC,
-  0x1C, 0x9C, 0x5C, 0xDC, 0x3C, 0xBC, 0x7C, 0xFC,
-  0x02, 0x82, 0x42, 0xC2, 0x22, 0xA2, 0x62, 0xE2,
-  0x12, 0x92, 0x52, 0xD2, 0x32, 0xB2, 0x72, 0xF2,
-  0x0A, 0x8A, 0x4A, 0xCA, 0x2A, 0xAA, 0x6A, 0xEA,
-  0x1A, 0x9A, 0x5A, 0xDA, 0x3A, 0xBA, 0x7A, 0xFA,
-  0x06, 0x86, 0x46, 0xC6, 0x26, 0xA6, 0x66, 0xE6,
-  0x16, 0x96, 0x56, 0xD6, 0x36, 0xB6, 0x76, 0xF6,
-  0x0E, 0x8E, 0x4E, 0xCE, 0x2E, 0xAE, 0x6E, 0xEE,
-  0x1E, 0x9E, 0x5E, 0xDE, 0x3E, 0xBE, 0x7E, 0xFE,
-  0x01, 0x81, 0x41, 0xC1, 0x21, 0xA1, 0x61, 0xE1,
-  0x11, 0x91, 0x51, 0xD1, 0x31, 0xB1, 0x71, 0xF1,
-  0x09, 0x89, 0x49, 0xC9, 0x29, 0xA9, 0x69, 0xE9,
-  0x19, 0x99, 0x59, 0xD9, 0x39, 0xB9, 0x79, 0xF9,
-  0x05, 0x85, 0x45, 0xC5, 0x25, 0xA5, 0x65, 0xE5,
-  0x15, 0x95, 0x55, 0xD5, 0x35, 0xB5, 0x75, 0xF5,
-  0x0D, 0x8D, 0x4D, 0xCD, 0x2D, 0xAD, 0x6D, 0xED,
-  0x1D, 0x9D, 0x5D, 0xDD, 0x3D, 0xBD, 0x7D, 0xFD,
-  0x03, 0x83, 0x43, 0xC3, 0x23, 0xA3, 0x63, 0xE3,
-  0x13, 0x93, 0x53, 0xD3, 0x33, 0xB3, 0x73, 0xF3,
-  0x0B, 0x8B, 0x4B, 0xCB, 0x2B, 0xAB, 0x6B, 0xEB,
-  0x1B, 0x9B, 0x5B, 0xDB, 0x3B, 0xBB, 0x7B, 0xFB,
-  0x07, 0x87, 0x47, 0xC7, 0x27, 0xA7, 0x67, 0xE7,
-  0x17, 0x97, 0x57, 0xD7, 0x37, 0xB7, 0x77, 0xF7,
-  0x0F, 0x8F, 0x4F, 0xCF, 0x2F, 0xAF, 0x6F, 0xEF,
-  0x1F, 0x9F, 0x5F, 0xDF, 0x3F, 0xBF, 0x7F, 0xFF
-};
-#endif  /* BROTLI_RBIT */
-
-#define BROTLI_REVERSE_BITS_LOWEST \
-  ((brotli_reg_t)1 << (BROTLI_REVERSE_BITS_MAX - 1 + BROTLI_REVERSE_BITS_BASE))
-
-/* Returns reverse(num >> BROTLI_REVERSE_BITS_BASE, BROTLI_REVERSE_BITS_MAX),
-   where reverse(value, len) is the bit-wise reversal of the len least
-   significant bits of value. */
-static BROTLI_INLINE brotli_reg_t BrotliReverseBits(brotli_reg_t num) {
-#if defined(BROTLI_RBIT)
-  return BROTLI_RBIT(num);
-#else
-  return kReverseBits[num];
-#endif
-}
-
-/* Stores code in table[0], table[step], table[2*step], ..., table[end] */
-/* Assumes that end is an integer multiple of step */
-static BROTLI_INLINE void ReplicateValue(HuffmanCode* table,
-                                         int step, int end,
-                                         HuffmanCode code) {
-  do {
-    end -= step;
-    table[end] = code;
-  } while (end > 0);
-}
-
-/* Returns the table width of the next 2nd level table. |count| is the histogram
-   of bit lengths for the remaining symbols, |len| is the code length of the
-   next processed symbol. */
-static BROTLI_INLINE int NextTableBitSize(const uint16_t* const count,
-                                          int len, int root_bits) {
-  int left = 1 << (len - root_bits);
-  while (len < BROTLI_HUFFMAN_MAX_CODE_LENGTH) {
-    left -= count[len];
-    if (left <= 0) break;
-    ++len;
-    left <<= 1;
-  }
-  return len - root_bits;
-}
-
-void BrotliBuildCodeLengthsHuffmanTable(HuffmanCode* table,
-                                        const uint8_t* const code_lengths,
-                                        uint16_t* count) {
-  HuffmanCode code;       /* current table entry */
-  int symbol;             /* symbol index in original or sorted table */
-  brotli_reg_t key;       /* prefix code */
-  brotli_reg_t key_step;  /* prefix code addend */
-  int step;               /* step size to replicate values in current table */
-  int table_size;         /* size of current table */
-  int sorted[BROTLI_CODE_LENGTH_CODES];  /* symbols sorted by code length */
-  /* offsets in sorted table for each length */
-  int offset[BROTLI_HUFFMAN_MAX_CODE_LENGTH_CODE_LENGTH + 1];
-  int bits;
-  int bits_count;
-  BROTLI_DCHECK(BROTLI_HUFFMAN_MAX_CODE_LENGTH_CODE_LENGTH <=
-                BROTLI_REVERSE_BITS_MAX);
-
-  /* Generate offsets into sorted symbol table by code length. */
-  symbol = -1;
-  bits = 1;
-  BROTLI_REPEAT(BROTLI_HUFFMAN_MAX_CODE_LENGTH_CODE_LENGTH, {
-    symbol += count[bits];
-    offset[bits] = symbol;
-    bits++;
-  });
-  /* Symbols with code length 0 are placed after all other symbols. */
-  offset[0] = BROTLI_CODE_LENGTH_CODES - 1;
-
-  /* Sort symbols by length, by symbol order within each length. */
-  symbol = BROTLI_CODE_LENGTH_CODES;
-  do {
-    BROTLI_REPEAT(6, {
-      symbol--;
-      sorted[offset[code_lengths[symbol]]--] = symbol;
-    });
-  } while (symbol != 0);
-
-  table_size = 1 << BROTLI_HUFFMAN_MAX_CODE_LENGTH_CODE_LENGTH;
-
-  /* Special case: all symbols but one have 0 code length. */
-  if (offset[0] == 0) {
-    code.bits = 0;
-    code.value = (uint16_t)sorted[0];
-    for (key = 0; key < (brotli_reg_t)table_size; ++key) {
-      table[key] = code;
-    }
-    return;
-  }
-
-  /* Fill in table. */
-  key = 0;
-  key_step = BROTLI_REVERSE_BITS_LOWEST;
-  symbol = 0;
-  bits = 1;
-  step = 2;
-  do {
-    code.bits = (uint8_t)bits;
-    for (bits_count = count[bits]; bits_count != 0; --bits_count) {
-      code.value = (uint16_t)sorted[symbol++];
-      ReplicateValue(&table[BrotliReverseBits(key)], step, table_size, code);
-      key += key_step;
-    }
-    step <<= 1;
-    key_step >>= 1;
-  } while (++bits <= BROTLI_HUFFMAN_MAX_CODE_LENGTH_CODE_LENGTH);
-}
-
-uint32_t BrotliBuildHuffmanTable(HuffmanCode* root_table,
-                                 int root_bits,
-                                 const uint16_t* const symbol_lists,
-                                 uint16_t* count) {
-  HuffmanCode code;       /* current table entry */
-  HuffmanCode* table;     /* next available space in table */
-  int len;                /* current code length */
-  int symbol;             /* symbol index in original or sorted table */
-  brotli_reg_t key;       /* prefix code */
-  brotli_reg_t key_step;  /* prefix code addend */
-  brotli_reg_t sub_key;   /* 2nd level table prefix code */
-  brotli_reg_t sub_key_step;  /* 2nd level table prefix code addend */
-  int step;               /* step size to replicate values in current table */
-  int table_bits;         /* key length of current table */
-  int table_size;         /* size of current table */
-  int total_size;         /* sum of root table size and 2nd level table sizes */
-  int max_length = -1;
-  int bits;
-  int bits_count;
-
-  BROTLI_DCHECK(root_bits <= BROTLI_REVERSE_BITS_MAX);
-  BROTLI_DCHECK(BROTLI_HUFFMAN_MAX_CODE_LENGTH - root_bits <=
-                BROTLI_REVERSE_BITS_MAX);
-
-  while (symbol_lists[max_length] == 0xFFFF) max_length--;
-  max_length += BROTLI_HUFFMAN_MAX_CODE_LENGTH + 1;
-
-  table = root_table;
-  table_bits = root_bits;
-  table_size = 1 << table_bits;
-  total_size = table_size;
-
-  /* Fill in the root table. Reduce the table size to if possible,
-     and create the repetitions by memcpy. */
-  if (table_bits > max_length) {
-    table_bits = max_length;
-    table_size = 1 << table_bits;
-  }
-  key = 0;
-  key_step = BROTLI_REVERSE_BITS_LOWEST;
-  bits = 1;
-  step = 2;
-  do {
-    code.bits = (uint8_t)bits;
-    symbol = bits - (BROTLI_HUFFMAN_MAX_CODE_LENGTH + 1);
-    for (bits_count = count[bits]; bits_count != 0; --bits_count) {
-      symbol = symbol_lists[symbol];
-      code.value = (uint16_t)symbol;
-      ReplicateValue(&table[BrotliReverseBits(key)], step, table_size, code);
-      key += key_step;
-    }
-    step <<= 1;
-    key_step >>= 1;
-  } while (++bits <= table_bits);
-
-  /* If root_bits != table_bits then replicate to fill the remaining slots. */
-  while (total_size != table_size) {
-    memcpy(&table[table_size], &table[0],
-           (size_t)table_size * sizeof(table[0]));
-    table_size <<= 1;
-  }
-
-  /* Fill in 2nd level tables and add pointers to root table. */
-  key_step = BROTLI_REVERSE_BITS_LOWEST >> (root_bits - 1);
-  sub_key = (BROTLI_REVERSE_BITS_LOWEST << 1);
-  sub_key_step = BROTLI_REVERSE_BITS_LOWEST;
-  for (len = root_bits + 1, step = 2; len <= max_length; ++len) {
-    symbol = len - (BROTLI_HUFFMAN_MAX_CODE_LENGTH + 1);
-    for (; count[len] != 0; --count[len]) {
-      if (sub_key == (BROTLI_REVERSE_BITS_LOWEST << 1U)) {
-        table += table_size;
-        table_bits = NextTableBitSize(count, len, root_bits);
-        table_size = 1 << table_bits;
-        total_size += table_size;
-        sub_key = BrotliReverseBits(key);
-        key += key_step;
-        root_table[sub_key].bits = (uint8_t)(table_bits + root_bits);
-        root_table[sub_key].value =
-            (uint16_t)(((size_t)(table - root_table)) - sub_key);
-        sub_key = 0;
-      }
-      code.bits = (uint8_t)(len - root_bits);
-      symbol = symbol_lists[symbol];
-      code.value = (uint16_t)symbol;
-      ReplicateValue(
-          &table[BrotliReverseBits(sub_key)], step, table_size, code);
-      sub_key += sub_key_step;
-    }
-    step <<= 1;
-    sub_key_step >>= 1;
-  }
-  return (uint32_t)total_size;
-}
-
-uint32_t BrotliBuildSimpleHuffmanTable(HuffmanCode* table,
-                                       int root_bits,
-                                       uint16_t* val,
-                                       uint32_t num_symbols) {
-  uint32_t table_size = 1;
-  const uint32_t goal_size = 1U << root_bits;
-  switch (num_symbols) {
-    case 0:
-      table[0].bits = 0;
-      table[0].value = val[0];
-      break;
-    case 1:
-      table[0].bits = 1;
-      table[1].bits = 1;
-      if (val[1] > val[0]) {
-        table[0].value = val[0];
-        table[1].value = val[1];
-      } else {
-        table[0].value = val[1];
-        table[1].value = val[0];
-      }
-      table_size = 2;
-      break;
-    case 2:
-      table[0].bits = 1;
-      table[0].value = val[0];
-      table[2].bits = 1;
-      table[2].value = val[0];
-      if (val[2] > val[1]) {
-        table[1].value = val[1];
-        table[3].value = val[2];
-      } else {
-        table[1].value = val[2];
-        table[3].value = val[1];
-      }
-      table[1].bits = 2;
-      table[3].bits = 2;
-      table_size = 4;
-      break;
-    case 3: {
-      int i, k;
-      for (i = 0; i < 3; ++i) {
-        for (k = i + 1; k < 4; ++k) {
-          if (val[k] < val[i]) {
-            uint16_t t = val[k];
-            val[k] = val[i];
-            val[i] = t;
-          }
-        }
-      }
-      for (i = 0; i < 4; ++i) {
-        table[i].bits = 2;
-      }
-      table[0].value = val[0];
-      table[2].value = val[1];
-      table[1].value = val[2];
-      table[3].value = val[3];
-      table_size = 4;
-      break;
-    }
-    case 4: {
-      int i;
-      if (val[3] < val[2]) {
-        uint16_t t = val[3];
-        val[3] = val[2];
-        val[2] = t;
-      }
-      for (i = 0; i < 7; ++i) {
-        table[i].value = val[0];
-        table[i].bits = (uint8_t)(1 + (i & 1));
-      }
-      table[1].value = val[1];
-      table[3].value = val[2];
-      table[5].value = val[1];
-      table[7].value = val[3];
-      table[3].bits = 3;
-      table[7].bits = 3;
-      table_size = 8;
-      break;
-    }
-  }
-  while (table_size != goal_size) {
-    memcpy(&table[table_size], &table[0],
-           (size_t)table_size * sizeof(table[0]));
-    table_size <<= 1;
-  }
-  return goal_size;
-}
-
-#if defined(__cplusplus) || defined(c_plusplus)
-}  /* extern "C" */
-#endif