REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4683
Add sample unit tests that always fail or generate unexpected
exceptions along with a new DSC file to build the unit tests
that always fail or generate unexpected exceptions. This can
be used to verify the log information on failures is accurate
and provides the correct information to determine the source
of the unit test failure.
Divide by zero is used to generate unexpected exceptions. The
compiler warnings for divide by zero are disables for the unit
tests that generate divide by zero exceptions on purpose.
These tests are not added to CI because CI would always fail.
The UnitTestFrameworkPkg.ci.yaml file is updated to ignore the
INF files for host-based testing that always fail.
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4683
Update GoogleTest samples to use EXPECT_ANY_THROW() instead
of ASSERT_DEATH(). ASSERT_DEATH() is a very slow method to
detect an expected ASSERT() condition. Throwing an exception
from ASSERT() and using EXPECT_ANY_THROW() is several orders
of magnitude faster.
Update GoogleTest sample with example of using EXPECT_THROW()
and EXPECT_THAT() to check for more specific ASSERT() conditions
that allow unit test cases to test functions that contain
more than one ASSERT() statement and verify that the expected
ASSERT() is the one that was actually triggered.
Update library mappings so target-based unit tests use
UnitTestDebugAssertLib.inf and host-based unit tests use
UnitTestDebugAssertLibHost.inf
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4683
Update GetActiveFrameworkHandle() to remove ASSERT() and require
caller to check for NULL.
This allows GetActiveFrameworkHandle() to be used to determine if the
current host-based test environment is framework/cmocka or gtest. In
the framework/cmocka host-based environment GetActiveFrameworkHandle()
returns non-NULL. In the gtest host-based environment
GetActiveFrameworkHandle() returns NULL.
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4683
Update MSFT CC_FLAGS for host-based unit tests to use /EHs
instead of /EHsc to support building C functions with SEH
(Structured Exception Handling) enabled. This is required to
build UnitTestDebugAssertLibHost.inf.
Update GCC CC_FLAGS for host-based unit tests to use -fexceptions
to support catching exceptions.
Update GoogleTestLib.h to include Throws() APIs that enable
unit tests to use EXPECT_THAT() to check for expected ASSERT()
conditions for a specific ASSERT() expression.
Update GCC CC_FLAGS to add --coverage for host-based builds
for all GCC tool chains.
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4683
Add /MT to MSFT CC_FLAGS to always use release libraries
when building host-based unit tests so any exceptions
generated during host-based test execution generate an
error message in stderr instead of a popup window.
Use /MTd when -D UNIT_TESTING_DEBUG is to use debug
libraries when building host-based unit tests so any
exceptions generated during host-based test execution
generate a popup window with option to attach a debugger.
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4610
Google Test hides test registration in global constructors on global
objects. Global constructors are traditionally implemented by placing
references to the global constructor's symbol in special sections
(traditionally named .ctors or .init_array). These sections are not
explicitly referenced by the linker, and libc only looks at special
start and end symbols (and calls them).
This works fine if you're linking a program manually using
gcc a.o b.o c.o -o test_suite
but fails miserably when using static libraries (such as what EDK2
does), because traditional static archive symbol resolution rules don't
allow for object files to be pulled in to the link if there isn't an
undefined symbol reference to that .o elsewhere.
Fix it by passing --whole-archive (GCC) and /WHOLEARCHIVE (MSVC). These
options force the linker to pull in the entire static library, thus
including previously-unreferenced constructors and making sure
multi-file gtest EDK2 components work.
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Pedro Falcato <pedro.falcato@gmail.com>
Reviewed-by: Cc: Michael D Kinney <michael.d.kinney@intel.com>
Adds a PrEval entry to the package's ci.yaml file which is used to
verify if the package uses a particular library instance when that
library instance file (INF) is updated.
When a library instance file (INF) is updated, PrEval will review each
package's DSC as described in the ci.yaml file to determine if the
package uses said library instance. If the package does use the library
instance, it will be built and tested to ensure the package is not
broken from the change.
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4467
Current implementation of UnitTestFrameworkPkg for shell-based unit test
will save the unit test cache to the same volume as the test application
itself. This works as long as the test application is on a writable
volume, such as USB or EFI partition.
Instead of saving the files to the same file system of unit test
application, this change will save the cache file to the path where the
user ran this test application.
This change then added an input argument to allow user to specify where
to save such cache file through `--CachePath` shell argument to allow
even more flexibility.
This change was tested on proprietary physical hardware platforms and
QEMU based virtual platform.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Kun Qin <kuqin12@gmail.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
This library supports a PeiServicesTablePointerLib implementation
that allows code dependent upon PeiServicesTable to operate in an
isolated execution environment such as within the context of a
host-based unit test framework.
The unit test should initialize the PeiServicesTable database with
any required elements (e.g. PPIs, Hob etc.) prior to the services
being invoked by code under test.
It is strongly recommended to clean any global databases by using
EFI_PEI_SERVICES.ResetSystem2 after every unit test so the tests
execute in a predictable manner from a clean state.
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
__FUNCTION__ is a pre-standard extension that gcc and Visual C++ among
others support, while __func__ was standardized in C99.
Since it's more standard, replace __FUNCTION__ with __func__ throughout
UnitTestFrameworkPkg.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
This library supports a Boot Services table library implementation
that allows code dependent upon UefiBootServicesTableLib to operate
in an isolated execution environment such as within
the context of a host-based unit test framework.
The unit test should initialize the Boot Services database with any
required elements (e.g. protocols, events, handles, etc.) prior to
the services being invoked by code under test.
It is strongly recommended to clean any global databases (e.g.
protocol, event, handles, etc.) after every unit test so the tests
execute in a predictable manner from a clean state.
This library is being moved here from PrmPkg so it can be made more
generally available to other packages and improved upon for others
use.
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
REF: REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2801
Add UnitTestDebugAssertLib that provides the UnitTestDebugAssert()
service and the gUnitTestExpectAssertFailureJumpBuffer global
variable. This NULL library is linked against all host and target
unit test builds. This guarantees that the UnitTestDebugAssert()
service is available to link against all libraries and modules that
use the DebugLib class.
EDKII_UNIT_TEST_FRAMEWORK_ENABLED must always be defined when
building unit tests so the behavior of the DebugLib ASSERT()
macros can be adjusted to allow the unit test framework to
catch an ASSERT() if it is triggered by a function under test.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2805
If a unit test fails with an exception or an assert, then the
CmockaUnitTestFunctionRunner() is terminated and the logic
that follows the invocation of the unit test is skipped. This
currently skips the logic that prints log messages.
Move the print of log messages to the end of the function
CmockaUnitTestTeardownFunctionRunner() that is guaranteed to
be executed when a unit test completes normally or is
terminated with an exception or an assert.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2801
The default DebugLib for target mode was DebugLibNull. This
library instance disables all ASSERT() and DEBUG() macros
which removes the ability to write unit tests that check for
ASSERT() behaviors.
The DebugLib is changed to PeiDxeDebugLibReportStatusCode.inf
that guarantees that DEBUG() and ASSERT() macros are active. The
default ReportStatusCodeLib is set to BaseReportStatusCodeLibNull.inf
so no messages are sent to any devices preserving the DebugLibNull
behavior.
A platform specific unit test can always override these mappings
with a platform specific DebugLib.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2804
Optionally enable a feature to support source level debug of a
host based unit test. By default, this feature is disabled.
Exceptions are caught by the unit test framework and are
interpreted as a test failure.
When a unit test is under development, bugs may generate
exceptions or a unit test developer may want to trace the
execution of unit tests to debug some unexpected behavior.
Defining UNIT_TESTING_DEBUG in the DSC file or from the build
command line allows exceptions to be caught by the host OS
and allows the developer to debug their unit test under
development or debug the Unit Test Framework itself.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
