1310 Commits

Author SHA1 Message Date
Carsey, Jaben
b298567268 BaseTools/build/build: refactor and move functions
Move DataDump and DataRestore from Common.Misc to this file.
There were no other consumers of these 2 functions.

Import threading since that module is used in build.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-21 18:03:30 +08:00
Fan, ZhijuX
dd1988cf01 BaseTools:build break if the Path contains SingleFile.Ext
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1325

if SingleFile.Path = "/foo/bar.Sap/yada/source.S" and
SingleFile.Ext = ".S". Then key would end up "/foo/bar"
instead of "/foo/bar.Sap/yada/source" as intended.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-21 18:03:30 +08:00
Felix Polyudov
842695d2ff BaseTools: Fix incorrect formatting of GenFds command dictionary
GenFdsCommand returned dictionary with elements that
are not compatible with GenFdsApi.
As a result the following options were not processed by GenFdsApi:
-v, -q, -d, --genfds-multi-thread, --ignore-sources
The issue is introduced by commit b3497bad1221704a5dbc5da0b10f42625f1ad2ed.

V2: Remove EFI_SOURCE references

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Felix Polyudov <felixp@ami.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-21 18:03:29 +08:00
Fan, ZhijuX
3e1ab49481 BaseTools:Build fail when PCD use in the [DEPEX] section of INF files
Update _FixedPcdVoidTypeDict to FixedVoidTypePcds
'_FixedPcdVoidTypeDict' no longer exists because edk2 version (b23414f6).
'ModuleAutoGen' object has no attribute '_FixedPcdVoidTypeDict'.
Build fail when PCD use in the [DEPEX] section of INF files

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-21 18:03:28 +08:00
Feng, Bob C
cce9d76358 BaseTools: Allow empty value for HiiPcd in Dsc
https://bugzilla.tianocore.org/show_bug.cgi?id=1466

DEC file defines PCD default value and PCD supported type.
DSC can configure PCD type and value.
If the value is same to default value in DEC file,
DSC can only configure PCD type and leave empty for value.
This usage supports all type PCD except for DynamicHii type.
So, DynamicHii PCD should support this usage. Below is one example in DSC.

for example,
[PcdsDynamicHii.common.DEFAULT]
PcdPkgTokenSpaceGuid.PcdCName|L"VarName"|gVarGuid|0x00||NV,BS

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-18 10:20:01 +08:00
Feng, Bob C
c68b564900 BaseTools: Remove EDK_SOURCE keyword from Inf Parser.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove unused EDK_SOURCE keyword from Inf Parser.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-17 19:49:20 +08:00
Feng, Bob C
7f98ef37f6 BaseTools: Remove EDK_SOURCE keyword from GenFds tool.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove unused EDK_SOURCE keyword from GenFds tool.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-17 19:49:20 +08:00
Feng, Bob C
0006d3fd0f BaseTools: Remove EDK_SOURCE keyword from ECC Tool
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove EDK_SOURCE keyword from ECC Tool.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-17 19:49:20 +08:00
Liming Gao
9e90fb097c BaseTools VolInfo: Fix XCODE5 build issue
XCODE5 reports warning "equality comparison with extraneous parentheses".
Remove extraneous parentheses around the comparison to silence this warning.
This issue is caused by commit 8daa4278e80c70e6caabc525cd122744488253f5.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Feng Bob C <bob.c.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2019-01-16 22:37:59 +08:00
Liming Gao
2d4fd226da BaseTools GenFw: Fix XCODE5 build issue
Warning message is that "equality comparison with extraneous parentheses"
Remove extraneous parentheses around the comparison to silence this warning.
This issue is caused by commit 8daa4278e80c70e6caabc525cd122744488253f5.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Feng Bob C <bob.c.feng@intel.com>
Reviewed-by: Feng Bob C <bob.c.feng@intel.com>
2019-01-15 14:44:01 +08:00
Feng, Bob C
4523bc82ae BaseTools: Remove unused logic for EDKI
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove gEcpSource global variable in python tool.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
2019-01-14 15:24:22 +08:00
Feng, Bob C
5bc96263ea BaseTools: Enable component override functionality
https://bugzilla.tianocore.org/show_bug.cgi?id=1449
This patch enable build tools to recognize that
when two given files have the same GUID, file path and ARCH in Dsc,
The later one's definition will be used.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Carsey Jaben <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-14 11:14:24 +08:00
Feng, Bob C
8daa4278e8 BaseTools: Remove unused logic from C tools
https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove IA64 support from BaseTools C code.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
2019-01-14 09:34:41 +08:00
Feng, Bob C
39879ef267 BaseTools: Remove unused logic for IPF
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove IPF support from BaseTools C code.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-14 09:34:41 +08:00
Feng, Bob C
8229250132 BaseTools: Remove unused logic for EDKI
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove EDK module type support from BaseTools python code.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-14 09:34:23 +08:00
Feng, Bob C
b8920ccc3c BaseTools: Report Error if use SET in Dsc
Build tool do not support SET syntax in DSC.
If the SET statement is used in DSC, build tool just ignore it.
That behavior confused some users that
they think SET statement works in DSC like in FDF.
To avoid such confusion, build tool report ERROR
if there is "SET" statement in Dsc file.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Cc: Liming Gao <liming.gao@intel.com>
2019-01-11 10:51:19 +08:00
Carsey, Jaben
938cf4c33a BaseTools: fix imports
1 - Some of these imports are cascaded from another file. Import them locally.
2 - Some of these imports are not used. Remove them.
3 - Some of these were missing the namespace used to import them.

These changes facilitate optimization of BaseTools:
https://bugzilla.tianocore.org/show_bug.cgi?id=42

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-10 22:01:11 +08:00
Dandan Bi
9248a4717a BaseTools/VfrCompile: report error for Integer overflow
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1415

When an integer constant specified in the .vfr file is
too large for the varstore field it is being used with,
the VFR compiler reports an overflow warning like this:
Test.vfr(693): WARNING: Overflow: Value 1024 is too large to
         store in a UINT8
    : String to UINT* Overflow
Since Warning does not break the build process,
and it is easy to miss it.
This patch is to update the code to report error and break
the build if meet this kind of issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Dandan Bi <dandan.bi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-08 09:59:30 +08:00
Ard Biesheuvel
8ef653aa5a BaseTools/GenFds: permit stripped MM_CORE_STANDALONE binaries
The standalone MM core is executed in place, and resides in a
separate execution context which may be space constrained.
Since code and data may be mapped with different attributes for
security reasons, the PE/COFF binary could have a section
alignment of 4 KB.

This means that any relocation data is not only useless, but it
will also take up 4 KB of valuable space.

So add support for the RELOCS_STRIPPED attribute on FFS files of
this type, so that we can get rid of the .reloc section altogether.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-04 09:42:17 +01:00
Feng, Bob C
672601cfcc BaseTools: Correct PcdArray value assigment statement
https://bugzilla.tianocore.org/show_bug.cgi?id=1410
BaseTools should not generate C structure array initial value
if the value is not specified with CODE style.

This patch is going to remove the incorrect initial value statement
and correct the Pcd Array value assignment statement.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-04 15:18:18 +08:00
Feng, Bob C
82bfbd392a BaseTools: Fixed build report issue.
This patch is going to fix the regression issue
by 72a1d77694d51914c0dd6aa97dbfa58634b0a4a5

After enable PCD array, the Pcd.OverrideStruct has a new
key, array index, but the build report is not changed correspondingly.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-04 15:11:03 +08:00
Chasel, Chiu
2bb4a7ca62 BaseTools/GenFv: Support SecCore and PeiCore in different FV
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1423

There is usage model that SecCore and PeiCore are in different FVs.
Update BaseTools to support this usage model.

Test: Verified on internal platform with the case SecCore and
PeiCore in different FVs and built/booted successfully.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Chasel Chiu <chasel.chiu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-12-28 14:29:42 +08:00
Feng, Bob C
abc4c3386a BaseTools: Reset FdsGlobalVariable
https://bugzilla.tianocore.org/show_bug.cgi?id=1418
This patch is going to fix a regression issue that is introduced
by commit b3497bad1221704a5dbc5da0b10f42625f1ad2ed.

Before commit b3497b, build launched a external GenFds.py to generate
Fd, so the global variable in GenFds.py was reset in each execution.

After commit b3497b, each GenFds run in the same python interpeter, so
we need to explicitly reset global variable in each GenFdsApi call.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
2018-12-25 10:40:10 +08:00
Derek Lin
370544d116 BaseTools: Fix GenFds error doesn't break build.
Fix a bug because of b3497bad1221704a5dbc5da0b10f42625f1ad2ed.
Before the patch, when GenFds fail, the build continue and return success.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Derek Lin <derek.lin2@hpe.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-12-19 13:24:18 +08:00
Zhiju.Fan
073a76e662 BaseTools: Add $(INC)-like support when compiling .nasm files
current edk2\BaseTools\Conf\build_rule.template, the compile of nasm
source files does not have the $(INC) support.

The '-I' option only includes the directory of the nasm source file
(${s_path}(+)). Hence, it will be impossible for nasm files to include
files outside of the nasm source file directory.

As a comparison, the compile of both .s and .asm have $(INC) support
in their compile commands.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=1085
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-12-19 08:42:14 +08:00
Feng, Bob C
643556fc48 BaseTools: Fixed metafile parser issues
https://bugzilla.tianocore.org/show_bug.cgi?id=1406
This patch is going to fix the regressions that
is introduced by commit 2f818ed0fb57d98985d151781a2ce9b8683129ee

The internal array for storing the metadata info should be cached
so that the meta file is parsed only once in one build.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-18 10:45:31 +08:00
Feng, Bob C
4c6e6f9f75 BaseTools: Fix PcdArray issue
https://bugzilla.tianocore.org/show_bug.cgi?id=1390

1. support hex number for array index
2. support Non-Dynamic Pcd for array data type
3. support {} and {CODE()} for array data type
4. Change GetStructurePcdMaxSize to be a static function since it need to
be called in another static function. And this function does not depend on
it's class instance.
5. Add unittest for RemoveCComments function and
ArrayIndex regular expression.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Philippe Mathieu-Daud? <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-18 10:15:50 +08:00
Feng, Bob C
48e890457b BaseTools: Fixed the build fail issue for cases
https://bugzilla.tianocore.org/show_bug.cgi?id=1386
This patch is going to fix the regression issue that is
introduced by commit 72a1d77694d51914c0dd6aa97dbfa58634b0a4a5

The issue will happen in the following cases:
1. There is no Pcd value assignment in Dsc file
2. There are duplicate Pcd filed assignment

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-15 20:16:46 +08:00
Feng, Bob C
9bf86b12ce BaseTools: Fixed bugs in CopyDict function
https://bugzilla.tianocore.org/show_bug.cgi?id=1387

This patch is going to fix the regression issue which is
introduced by commit bf9e636605188e291d33ab694ff1c5926b6f0800.

This patch Remove the CopyDict incorrect usage for non-dict
input data. Add a check for CopyDict input.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-15 20:13:32 +08:00
Feng, Bob C
5695877ec8 BaseTools: Fix PcdNvStoreDefaultValueBuffer Value.
https://bugzilla.tianocore.org/show_bug.cgi?id=1385
This patch is going to fix the regression issue that is
introduced by commit e6eae3b4c7b9b756263ecec79694de5f1e85b73a
and commit 0b6c5954e1d9a17e01eee7d5ef840a5b4790e2e8.

PcdNvStoreDefaultValueBuffer value is update to Vpd Info File,
but it is not update into a internal cache. This patch will
fix this incorrect value in that internal cache.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-15 15:45:07 +08:00
Ard Biesheuvel
0d68ce514b BaseTools/GenFw ARM: don't permit R_ARM_GOT_PREL relocations
We currently permit R_ARM_GOT_PREL relocations in the ELF32 conversion
routines, under the assumption that relative relocations are fine as
long as the section layout is the same between ELF and PE/COFF.

However, as is the case with any proxy generating relocation, it is
up to the linker to emit an entry in the GOT table and populate it
with the correct absolute address, which should also be fixed up at
PE/COFF load time. Unfortunately, the relocations covering the GOT
section are not emitted into the static relocation sections processed
by GenFw, but only in the dynamic relocation section as a R_ARM_RELATIVE
relocation, and so GenFw fails to emit the correct PE/COFF relocation
data for GOT entries.

Since GOT indirection is pointless anyway for PE/COFF modules running
in UEFI context, let's just drop the references to R_ARM_GOT_PREL from
GenFw, resulting in a build time failure rather than a runtime failure
if such relocations do occur.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-12 08:36:59 +01:00
BobCF
9601046bf4 BaseTools: Correct CCFLAG for PcdValueInit
https://bugzilla.tianocore.org/show_bug.cgi?id=1361
This patch is going to correct the CCFlag
for building PcdValueInit

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: BobCF <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-07 10:39:52 +08:00
BobCF
bf9e636605 BaseTools: Customize deepcopy function.
https://bugzilla.tianocore.org/show_bug.cgi?id=1288

This patch is one of build tool performance improvement
series patches.

This patch is going to customize the deepcopy function for
SkuClass, PcdClassObject and python dictionary.

python deepcopy copy everything of a object, but for our current
usage we just need to copy the data we care about recursively.

By implementing __deepcopy__ for SkuClass, PcdClassObject, we can customize
deepcopy function for them.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: BobCF <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-07 10:31:04 +08:00
BobCF
4e37570739 BaseTools: Optimize string concatenation
https://bugzilla.tianocore.org/show_bug.cgi?id=1288

This patch is one of build tool performance improvement
series patches.

This patch is going to use join function instead of
string += string2 statement.

Current code use string += string2 in a loop to combine
a string. while creating a string list in a loop and using
"".join(stringlist) after the loop will be much faster.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: BobCF <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-12-07 10:30:21 +08:00
BobCF
2f818ed0fb BaseTools: Replace the sqlite database with list
https://bugzilla.tianocore.org/show_bug.cgi?id=1288

[V2]
Optimize this patch so that it can be easy to review.
This patch is just apply the change to original files while
not create new similar files.

[V1]
This patch is one of build tool performance improvement
series patches.

This patch is going to use python list to store the parser data
instead of using sqlite database.

The replacement solution is as below:

SQL insert: list.append()
SQL select: list comprehension. for example:
Select * from table where field = “something”
->
[ item for item in table if item[3] == “something”]

SQL update: python map function. for example:
Update table set field1=newvalue where filed2 = “something”.
-> map(lambda x: x[1] = newvalue,
   [item for item in table if item[2] == “something”])

SQL delete: list comprehension.

With this change, We can save the time of interpreting SQL statement
and the time of write database to file system

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: BobCF <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-12-07 10:23:07 +08:00
Zhao, ZhiqiangX
b3497bad12 BaseTools: AutoGen and GenFds share the parser data.
V2:
Extract the common part of new API and the original main() function
into one function.

V1:
https://bugzilla.tianocore.org/show_bug.cgi?id=1288

Currently, AutoGen and GenFds run in different python interpreters. The
parser are duplicated. This patch is going to create new API for GenFds
and have the build to call that API instead of executing GenFds.py. As
such, the GenFds and build can share the parser data.

This patch is expected to save the time of GenFds about 2~3 seconds.
More details will be logged in BZ.

This is the summary measure data generated from python cProfile for
building Ovmf.

Currently:
8379147 function calls (8135450 primitive calls) in 12.580 seconds

After applying this patch:
3428712 function calls (3418881 primitive calls) in 8.944 seconds

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Carsey Jaben <jaben.carsey@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-12-07 10:12:05 +08:00
Feng, Bob C
34e733f200 BaseTool: Filter out unused structure pcds
V2:
Fixed the issue that V1 adds new check
to the Pcds in the platform unused library INF files.
It breaks the existing platform.

V1?
The current code handle all the structure pcds
even if there is no module or library use them.
This patch is going to filter out the unused structure pcds.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-07 10:03:02 +08:00
bob.c.feng@intel.com
72a1d77694 BaseTools: Enable Pcd Array support.
https://bugzilla.tianocore.org/show_bug.cgi?id=1292

This patch is going to enable Array data type for PCD.

1. Support Pcd ARRAY as Structure PCD type
   including basic datatype array and structure array.
   For example:
   gStructuredPcdPkgTokenSpaceGuid.PcdTest|{0x0}|TEST[10]|0x00010080
   gStructuredPcdPkgTokenSpaceGuid.PcdTest2|{0x0}|UINT8[10]|0x00010081
2. Support C CODE style value initialization in DEC/DSC.
   For example:
gStructuredPcdPkgTokenSpaceGuid.PcdTest|{CODE({
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
  {0, {0, 0, 0, 0,  0, 0, 0}},
})}

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-07 09:59:36 +08:00
Jaben Carsey
bc39c5cbea BaseTools: create and use a standard shared variable for '*'
add a variable for the string '*' and then use it instead of lots of '*'

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by : Bob Feng <bob.c.feng@intel.com>
2018-12-06 17:11:24 +08:00
Jaben Carsey
ea55e37c36 BaseTools: cleanup LongFilePathSupport usage
1) remove an identical function and import it from Common.LongFilePathSupport
2) remove an import that is not needed/used.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by : Bob Feng <bob.c.feng@intel.com>
2018-12-06 17:10:27 +08:00
Jaben Carsey
4495792953 BaseTools: Move Identification file to Eot
Move the Identification file.
This file is only ever imported into the Eot tool.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by : Bob Feng <bob.c.feng@intel.com>
2018-12-06 17:08:09 +08:00
Ard Biesheuvel
67938bcc9d BaseTools/CommonLib: drop the use of MAX_ADDRESS
The macro MAX_ADDRESS represents the largest virtual address that
is valid for a certain architecture. For the BaseTools, this quantity
is irrelevant, since the same tools can be used to build for different
targets.

Since we only refer to it in a single place, which is an ASSERT() that
doesn't seem particularly useful (it ensures that memcpy() will not
be called with arguments that will make it read beyond the end of the
address space and wrap around), let's drop the ASSERT and all references
to MAX_ADDRESS.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-06 08:32:00 +01:00
Shenglei Zhang
7702ceb8af BaseTools: Remove tools only used by DuetPkg
Given that DuetPkg will be removed, tools only used by
DuetPkg can also be removed after its removal operation.
https://bugzilla.tianocore.org/show_bug.cgi?id=1322

v2:Remove these tools in Makefile and GNUmakefile.

v4:Remove these tools in BinWrappers/PosixLike/ and
   UserManuals.

Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Hao Wu <hao.a.wu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>
Reviewed-by: Hao Wu <hao.a.wu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-06 14:32:04 +08:00
Ard Biesheuvel
8efc6d84ca BaseTools/CommonLib: drop definition of MAX_UINTN
The maximum value that can be represented by the native word size
of the *target* should be irrelevant when compiling tools that
run on the build *host*. So drop the definition of MAX_UINTN, now
that we no longer use it.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:04:54 +01:00
Ard Biesheuvel
98a9519f1c BaseTools/CommonLib: get rid of 'native' type string parsing routines
Parsing a string into an integer variable of the native word size
is not defined for the BaseTools, since the same tools may be used
to build firmware for different targets with different native word
sizes.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:04:54 +01:00
Ard Biesheuvel
c0b7379a31 BaseTools/DevicePath: use MAX_UINT32 as default device path max size
Replace the default size limit of IsDevicePathValid() with a value
that does not depend on the native word size of the build host.

4 GiB seems sufficient as the upper bound of a device path handled
by UEFI.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:04:54 +01:00
Ard Biesheuvel
4d393eb876 BaseTools/CommonLib: add definition of MAX_UINT32
Since we will be dropping the definition of MAX_UINTN, whose meaning
is ambiguous for the BaseTools, add a definition of MAX_UINT32 that
we can switch to.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:04:54 +01:00
Ard Biesheuvel
6e2d15e3c4 BaseTools/DevicePath: use explicit 64-bit number parsing routines
Replace invocations of StrHexToUintn() with StrHexToUint64(), so
that we can drop the former.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:03:23 +01:00
Ard Biesheuvel
76e5f493d2 BaseTools/CommonLib: use explicit 64-bit type in Strtoi()
Don't use the native word size string to number parsing routines,
but instead, use the 64-bit one and cast to UINTN.

Currently, the only user is in Source/C/DevicePath/DevicePathFromText.c
which takes care to use Strtoi64 () unless it assumes the value fits
in 32-bit, so this change is a no-op even on 32-bit build hosts.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:03:23 +01:00
Ard Biesheuvel
93afbd1595 BaseTools/CommonLib: avoid using 'native' word size in IP address handling
In the context of the BaseTools, there is no such thing as a native word
size, given that the same set of tools may be used to build a firmware
image consisting of both 32-bit and 64-bit modules.

So update StrToIpv4Address() and StrToIpv6Address() to use UINT64
types instead of UINTN types when parsing strings.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-05 09:03:17 +01:00