CreateStringTable and TempEntities both read their payload length as a fixed
width field on protocol 23 and older and as a var int after that, but the
encoders always write the var int. Reencoding an older demo therefore hands
the game a length it can't read: string tables come back as "bogus substring
index", and any packet carrying temp entities is malformed from there on,
taking whatever messages follow it in that packet with it.
The threshold and both widths are taken from the read paths a few lines up in
these same two files rather than worked out separately, so the two sides can't
drift: > 23, with 20 bits for CreateStringTable and 17 for TempEntities. The
engine agrees, for what it's worth - netmessages.cpp reads both of these under
> PROTOCOL_VERSION_23 at NET_MAX_PAYLOAD_BITS_V23 + 3 and
NET_MAX_PAYLOAD_BITS_V23 respectively, and that constant is 17.
serverinfo and prefetch already branch on the protocol on both sides, these
two were the only ones that didn't.
On a protocol 15 demo that was 28909 of 92518 reencoded packets failing to
parse back; on test_data/protocol23.dem, 13220 of 43327. Both are zero now,
so protocol23.dem can join the reencode round trip test. Protocol 24 output
is unchanged, byte for byte.
DemoHandler::default() starts out at protocol 24 and relies on handle_header
to replace that with whatever the demo actually is. Parsing through DemoParser
does this for you, but reencode drives the handler itself and never did, so
every demo got read as protocol 24.
Older protocols put a few fields at different widths, so the message stream
desyncs partway in. RawPacketStream reports that as a truncated demo rather
than an error, which is why nothing complains about it: reencode_demo on
test_data/protocol23.dem writes 1076 bytes, prints a truncation warning and
still exits 0.
The reencode test missed this because protocol23.dem was never in its list,
and because nothing checked the demo had been read to the end. Add a test for
that and the same assert to the existing round trip.
Output for protocol 24 demos is unchanged, byte for byte.
This is a follow up to PR #4. Handling for the Vector prop needs to be
fixed on the write/encoding side as well. When the float definition is
NormalFloatVar we need to skip writing the last component (z) to the
stream, since it's calculated from the first two components (x, y), and
just write out its sign.
SayText2 messages can use a simplified format where the body is just
client, raw, and a single null-terminated colored text string, without
the usual kind/from/text fields. The parser detected this by peeking
the first byte after client and raw and checking if it was \x01 (color
code for default color), but the text can also start with other color
codes such as \x07 (6-char hex color). When this happened, the parser
would consume the entire message text as the kind string (falling back
to ChatAll), leaving the text and from fields empty.
When the float definition is NormalFloatVar, the last component (z)
should be skipped and calculated from the first two components (x and
y), otherwise we can read too many bits, causing decoding failures:
* 744d6eb003/src/engine/dt_encode.cpp (L119-L143)
This is what was causing the issue with demos recorded on the
jump_classics_a3 map:
* https://codeberg.org/demostf/parser/issues/2
- Fix overflow in ExtraData by casting byte_len to usize before
multiplication. This was previously multiplying a u16 with
saturating_mul and would overflow when the extra data in string
tables reaches a certain size.
- Add test case for parsing large string table to verify correct
stream position after read.
