vasilito
cb424d7448
verify-patch-sanity.py validates every active recipe .patch has internally- consistent hunk line counts — catching the 'malformed patch at line N' failure at commit/CI/preflight time instead of hours into a cook. This cycle hit that class three times (qtwaylandscanner, sddm, xwayland), each only discovered when cookbook tried to apply the patch. Running it across the repo found 29 latent malformed patches (validated against GNU patch: e.g. relibc/P3-sysv-ipc reproduces 'malformed patch at line 22'). They were harmless only because they sit in vendored recipes (baked, not re- applied) — but would fail on any version-bump re-derivation. --fix recounts the hunk headers (body untouched) and repaired all 29. Wired into build-preflight.sh (Phase 1.0D) and redbear-ci.yml, with a unit test (test-patch-sanity.sh). Skips archived/legacy trees and unvalidatable formats (empty placeholders, bare-@@ git hunks).
316 lines
13 KiB
ReStructuredText
316 lines
13 KiB
ReStructuredText
=====================================
|
|
The PDB TPI and IPI Streams
|
|
=====================================
|
|
|
|
.. contents::
|
|
:local:
|
|
|
|
.. _tpi_intro:
|
|
|
|
Introduction
|
|
============
|
|
|
|
The PDB TPI Stream (Index 2) and IPI Stream (Index 4) contain information about
|
|
all types used in the program. It is organized as a :ref:`header <tpi_header>`
|
|
followed by a list of :doc:`CodeView Type Records <CodeViewTypes>`. Types are
|
|
referenced from various streams and records throughout the PDB by their
|
|
:ref:`type index <type_indices>`. In general, the sequence of type records
|
|
following the :ref:`header <tpi_header>` forms a topologically sorted DAG
|
|
(directed acyclic graph), which means that a type record B can only refer to
|
|
the type A if ``A.TypeIndex < B.TypeIndex``. While there are rare cases where
|
|
this property will not hold (particularly when dealing with object files
|
|
compiled with MASM), an implementation should try very hard to make this
|
|
property hold, as it means the entire type graph can be constructed in a single
|
|
pass.
|
|
|
|
.. important::
|
|
Type records form a topologically sorted DAG (directed acyclic graph).
|
|
|
|
.. _tpi_ipi:
|
|
|
|
TPI vs IPI Stream
|
|
=================
|
|
|
|
Recent versions of the PDB format (aka all versions covered by this document)
|
|
have 2 streams with identical layout, henceforth referred to as the TPI stream
|
|
and IPI stream. Subsequent contents of this document describing the on-disk
|
|
format apply equally whether it is for the TPI Stream or the IPI Stream. The
|
|
only difference between the two is in *which* CodeView records are allowed to
|
|
appear in each one, summarized by the following table:
|
|
|
|
+----------------------+---------------------+
|
|
| TPI Stream | IPI Stream |
|
|
+======================+=====================+
|
|
| LF_POINTER | LF_FUNC_ID |
|
|
+----------------------+---------------------+
|
|
| LF_MODIFIER | LF_MFUNC_ID |
|
|
+----------------------+---------------------+
|
|
| LF_PROCEDURE | LF_BUILDINFO |
|
|
+----------------------+---------------------+
|
|
| LF_MFUNCTION | LF_SUBSTR_LIST |
|
|
+----------------------+---------------------+
|
|
| LF_LABEL | LF_STRING_ID |
|
|
+----------------------+---------------------+
|
|
| LF_ARGLIST | LF_UDT_SRC_LINE |
|
|
+----------------------+---------------------+
|
|
| LF_FIELDLIST | LF_UDT_MOD_SRC_LINE |
|
|
+----------------------+---------------------+
|
|
| LF_ARRAY | |
|
|
+----------------------+---------------------+
|
|
| LF_CLASS | |
|
|
+----------------------+---------------------+
|
|
| LF_STRUCTURE | |
|
|
+----------------------+---------------------+
|
|
| LF_INTERFACE | |
|
|
+----------------------+---------------------+
|
|
| LF_UNION | |
|
|
+----------------------+---------------------+
|
|
| LF_ENUM | |
|
|
+----------------------+---------------------+
|
|
| LF_TYPESERVER2 | |
|
|
+----------------------+---------------------+
|
|
| LF_VFTABLE | |
|
|
+----------------------+---------------------+
|
|
| LF_VTSHAPE | |
|
|
+----------------------+---------------------+
|
|
| LF_BITFIELD | |
|
|
+----------------------+---------------------+
|
|
| LF_METHODLIST | |
|
|
+----------------------+---------------------+
|
|
| LF_PRECOMP | |
|
|
+----------------------+---------------------+
|
|
| LF_ENDPRECOMP | |
|
|
+----------------------+---------------------+
|
|
|
|
The usage of these records is described in more detail in
|
|
:doc:`CodeView Type Records <CodeViewTypes>`.
|
|
|
|
.. _type_indices:
|
|
|
|
Type Indices
|
|
============
|
|
|
|
A type index is a 32-bit integer that uniquely identifies a type inside of an
|
|
object file's ``.debug$T`` section or a PDB file's TPI or IPI stream. The
|
|
value of the type index for the first type record from the TPI stream is given
|
|
by the ``TypeIndexBegin`` member of the :ref:`TPI Stream Header <tpi_header>`
|
|
although in practice this value is always equal to 0x1000 (4096).
|
|
|
|
Any type index with a high bit set is considered to come from the IPI stream,
|
|
although this appears to be more of a hack, and LLVM does not generate type
|
|
indices of this nature. They can, however, be observed in Microsoft PDBs
|
|
occasionally, so one should be prepared to handle them. Note that having the
|
|
high bit set is not a necessary condition to determine whether a type index
|
|
comes from the IPI stream, it is only sufficient.
|
|
|
|
Once the high bit is cleared, any type index >= ``TypeIndexBegin`` is presumed
|
|
to come from the appropriate stream, and any type index less than this is a
|
|
bitmask which can be decomposed as follows:
|
|
|
|
.. code-block:: none
|
|
|
|
.---------------------------.------.----------.
|
|
| Unused | Mode | Kind |
|
|
'---------------------------'------'----------'
|
|
|+32 |+12 |+8 |+0
|
|
|
|
|
|
- **Kind** - A value from the following enum:
|
|
|
|
.. code-block:: c++
|
|
|
|
enum class SimpleTypeKind : uint32_t {
|
|
None = 0x0000, // uncharacterized type (no type)
|
|
Void = 0x0003, // void
|
|
NotTranslated = 0x0007, // type not translated by cvpack
|
|
HResult = 0x0008, // OLE/COM HRESULT
|
|
|
|
SignedCharacter = 0x0010, // 8 bit signed
|
|
UnsignedCharacter = 0x0020, // 8 bit unsigned
|
|
NarrowCharacter = 0x0070, // really a char
|
|
WideCharacter = 0x0071, // wide char
|
|
Character16 = 0x007a, // char16_t
|
|
Character32 = 0x007b, // char32_t
|
|
Character8 = 0x007c, // char8_t
|
|
|
|
SByte = 0x0068, // 8 bit signed int
|
|
Byte = 0x0069, // 8 bit unsigned int
|
|
Int16Short = 0x0011, // 16 bit signed
|
|
UInt16Short = 0x0021, // 16 bit unsigned
|
|
Int16 = 0x0072, // 16 bit signed int
|
|
UInt16 = 0x0073, // 16 bit unsigned int
|
|
Int32Long = 0x0012, // 32 bit signed
|
|
UInt32Long = 0x0022, // 32 bit unsigned
|
|
Int32 = 0x0074, // 32 bit signed int
|
|
UInt32 = 0x0075, // 32 bit unsigned int
|
|
Int64Quad = 0x0013, // 64 bit signed
|
|
UInt64Quad = 0x0023, // 64 bit unsigned
|
|
Int64 = 0x0076, // 64 bit signed int
|
|
UInt64 = 0x0077, // 64 bit unsigned int
|
|
Int128Oct = 0x0014, // 128 bit signed int
|
|
UInt128Oct = 0x0024, // 128 bit unsigned int
|
|
Int128 = 0x0078, // 128 bit signed int
|
|
UInt128 = 0x0079, // 128 bit unsigned int
|
|
|
|
Float16 = 0x0046, // 16 bit real
|
|
Float32 = 0x0040, // 32 bit real
|
|
Float32PartialPrecision = 0x0045, // 32 bit PP real
|
|
Float48 = 0x0044, // 48 bit real
|
|
Float64 = 0x0041, // 64 bit real
|
|
Float80 = 0x0042, // 80 bit real
|
|
Float128 = 0x0043, // 128 bit real
|
|
|
|
Complex16 = 0x0056, // 16 bit complex
|
|
Complex32 = 0x0050, // 32 bit complex
|
|
Complex32PartialPrecision = 0x0055, // 32 bit PP complex
|
|
Complex48 = 0x0054, // 48 bit complex
|
|
Complex64 = 0x0051, // 64 bit complex
|
|
Complex80 = 0x0052, // 80 bit complex
|
|
Complex128 = 0x0053, // 128 bit complex
|
|
|
|
Boolean8 = 0x0030, // 8 bit boolean
|
|
Boolean16 = 0x0031, // 16 bit boolean
|
|
Boolean32 = 0x0032, // 32 bit boolean
|
|
Boolean64 = 0x0033, // 64 bit boolean
|
|
Boolean128 = 0x0034, // 128 bit boolean
|
|
};
|
|
|
|
- **Mode** - A value from the following enum:
|
|
|
|
.. code-block:: c++
|
|
|
|
enum class SimpleTypeMode : uint32_t {
|
|
Direct = 0, // Not a pointer
|
|
NearPointer = 1, // Near pointer
|
|
FarPointer = 2, // Far pointer
|
|
HugePointer = 3, // Huge pointer
|
|
NearPointer32 = 4, // 32 bit near pointer
|
|
FarPointer32 = 5, // 32 bit far pointer
|
|
NearPointer64 = 6, // 64 bit near pointer
|
|
NearPointer128 = 7 // 128 bit near pointer
|
|
};
|
|
|
|
Note that for pointers, the bitness is represented in the mode. So a ``void*``
|
|
would have a type index with ``Mode=NearPointer32, Kind=Void`` if built for
|
|
32-bits but a type index with ``Mode=NearPointer64, Kind=Void`` if built for
|
|
64-bits.
|
|
|
|
By convention, the type index for ``std::nullptr_t`` is constructed the same
|
|
way as the type index for ``void*``, but using the bitless enumeration value
|
|
``NearPointer``.
|
|
|
|
.. _tpi_header:
|
|
|
|
Stream Header
|
|
=============
|
|
At offset 0 of the TPI Stream is a header with the following layout:
|
|
|
|
.. code-block:: c++
|
|
|
|
struct TpiStreamHeader {
|
|
uint32_t Version;
|
|
uint32_t HeaderSize;
|
|
uint32_t TypeIndexBegin;
|
|
uint32_t TypeIndexEnd;
|
|
uint32_t TypeRecordBytes;
|
|
|
|
uint16_t HashStreamIndex;
|
|
uint16_t HashAuxStreamIndex;
|
|
uint32_t HashKeySize;
|
|
uint32_t NumHashBuckets;
|
|
|
|
int32_t HashValueBufferOffset;
|
|
uint32_t HashValueBufferLength;
|
|
|
|
int32_t IndexOffsetBufferOffset;
|
|
uint32_t IndexOffsetBufferLength;
|
|
|
|
int32_t HashAdjBufferOffset;
|
|
uint32_t HashAdjBufferLength;
|
|
};
|
|
|
|
- **Version** - A value from the following enum.
|
|
|
|
.. code-block:: c++
|
|
|
|
enum class TpiStreamVersion : uint32_t {
|
|
V40 = 19950410,
|
|
V41 = 19951122,
|
|
V50 = 19961031,
|
|
V70 = 19990903,
|
|
V80 = 20040203,
|
|
};
|
|
|
|
Similar to the :doc:`PDB Stream <PdbStream>`, this value always appears to be
|
|
``V80``, and no other values have been observed. It is assumed that should
|
|
another value be observed, the layout described by this document may not be
|
|
accurate.
|
|
|
|
- **HeaderSize** - ``sizeof(TpiStreamHeader)``
|
|
|
|
- **TypeIndexBegin** - The numeric value of the type index representing the
|
|
first type record in the TPI stream. This is usually the value 0x1000 as
|
|
type indices lower than this are reserved (see :ref:`Type Indices
|
|
<type_indices>` for
|
|
a discussion of reserved type indices).
|
|
|
|
- **TypeIndexEnd** - One greater than the numeric value of the type index
|
|
representing the last type record in the TPI stream. The total number of
|
|
type records in the TPI stream can be computed as ``TypeIndexEnd -
|
|
TypeIndexBegin``.
|
|
|
|
- **TypeRecordBytes** - The number of bytes of type record data following the
|
|
header.
|
|
|
|
- **HashStreamIndex** - The index of a stream which contains a list of hashes
|
|
for every type record. This value may be -1, indicating that hash
|
|
information is not present. In practice a valid stream index is always
|
|
observed, so any producer implementation should be prepared to emit this
|
|
stream to ensure compatibility with tools which may expect it to be present.
|
|
|
|
- **HashAuxStreamIndex** - Presumably the index of a stream which contains a
|
|
separate hash table, although this has not been observed in practice and it's
|
|
unclear what it might be used for.
|
|
|
|
- **HashKeySize** - The size of a hash value (usually 4 bytes).
|
|
|
|
- **NumHashBuckets** - The number of buckets used to generate the hash values
|
|
in the aforementioned hash streams.
|
|
|
|
- **HashValueBufferOffset / HashValueBufferLength** - The offset and size within
|
|
the TPI Hash Stream of the list of hash values. It should be assumed that
|
|
there are either 0 hash values, or a number equal to the number of type
|
|
records in the TPI stream (``TypeIndexEnd - TypeEndBegin``). Thus, if
|
|
``HashBufferLength`` is not equal to ``(TypeIndexEnd - TypeEndBegin) *
|
|
HashKeySize`` we can consider the PDB malformed.
|
|
|
|
- **IndexOffsetBufferOffset / IndexOffsetBufferLength** - The offset and size
|
|
within the TPI Hash Stream of the Type Index Offsets Buffer. This is a list
|
|
of pairs of uint32_t's where the first value is a :ref:`Type Index
|
|
<type_indices>` and the second value is the offset in the type record data of
|
|
the type with this index. This can be used to do a binary search followed by
|
|
a linear search to get O(log n) lookup by type index.
|
|
|
|
- **HashAdjBufferOffset / HashAdjBufferLength** - The offset and size within
|
|
the TPI hash stream of a serialized hash table whose keys are the hash values
|
|
in the hash value buffer and whose values are type indices. This appears to
|
|
be useful in incremental linking scenarios, so that if a type is modified an
|
|
entry can be created mapping the old hash value to the new type index so that
|
|
a PDB file consumer can always have the most up to date version of the type
|
|
without forcing the incremental linker to garbage collect and update
|
|
references that point to the old version to now point to the new version.
|
|
The layout of this hash table is described in :doc:`HashTable`.
|
|
|
|
.. _tpi_records:
|
|
|
|
CodeView Type Record List
|
|
=========================
|
|
Following the header, there are ``TypeRecordBytes`` bytes of data that
|
|
represent a variable length array of :doc:`CodeView type records
|
|
<CodeViewTypes>`. The number of such records (e.g. the length of the array)
|
|
can be determined by computing the value ``Header.TypeIndexEnd -
|
|
Header.TypeIndexBegin``.
|
|
|
|
O(log(n)) access is provided by way of the Type Index Offsets array (if
|
|
present) described previously.
|