perl5260cdelta - what is new for cperl v5.26.0
This document describes the differences between the cperl 5.24.2 and the cperl 5.26.0 releases.
If you are upgrading from an earlier release such as v5.24.1c, first read the perl5242cdelta documentation, which describes differences between v5.24.2c and v5.24.1c.
cperl v5.26.0c was merged with perl v5.26.0 (as all previous major cperl releases). The rejected upstream commits for the differences have been documented at the github issues [cperl #165] and [cperl #256].
${^ENCODING} and the encoding pragma was not removed, rather fixed instead.
User-type support is greatly enhanced. See "Type-check assignments", "HvCLASS", "Type-infer bless", and "Type-infer subroutine return types". Before just coretypes were properly checked, now use types adds warnings for all other types.
The still incomplete and slow implementation for the experimental subroutine signatures feature from 5.25.4 was not added, as cperl's signatures are over 50% faster for over a year already and have many more features. In detail the new OP_ARGELEM, OP_ARGDEFELEM and OP_ARGCHECK are not used, cperl still uses a single OP_SIGNATURE op and passes its arguments properly as in XS on the stack, not via @_.
cperl doesn't use the slow Siphash 1-3 as default on 64bit, and no hybrid hash function as introduced with 5.25.8. cperl rather uses a short and fast hash function and other typical hash table optimizations, while adding proper security in the collision resolution instead. A secure PRF (pseudo random function) can never ensure DoS safety for a hash table, contrary to the Siphash paper claims.
cperl silently forbids attaching magic to the four major builtin SV sentinels undef, yes, no and placeholder, which are mostly compared to by pointer. Adding magic to them will break that comparison.
Assignment type violations are now also warned, with use warnings 'types' enabled, previously only signature types were checked. Only signature type violations or use types 'strict' violations are fatal.
Note that the type system is still completely unsound. So far it is only there to catch the most common errors and enable coretype optimizations. cperl only.
With cperl use base or use fields now closes the @ISA and hereby enable compile-time checks and optimizations. The new Internals::HvCLASS function gets or sets the same type for base/field classes as with the upcoming class keyword. See [cperl #249]. cperl only.
bless with a constant 2nd argument, the classname, infers this type to the enclosing sub if its the last statement in a body, or to the left-side assignment of a lexical variable. cperl only.
Subroutine types, either declared or inferred, are now passed through to the type-checker at compile-time. cperl only.
perl5.14 deprecated and 5.18 started disallowing a for loop with a qw() list, "qw-as-parens".
The rationale to remove the handy for qw() syntax was technical and trivial to fix. cperl 5.25.3 re-instated it for for loops, but not for the rest. cperl does not insist on the backwards syntax to require (qw( ... )) around the for list.
cperl5.25.3 -e'for qw(a b c) { print $_ }'
perl5.18 -e'for (qw(a b c)) { print $_ }'
perl5.14 -e'for $_ qw(a b c) { print $_ }'
=> Use of qw(...) as parentheses is deprecated at -e line 1
perl5.12 -e'for $_ qw(a b c) { print $_ }'The new additional cperl syntax is even easier to use than before. See [cperl #26]. cperl only.
Some platforms natively do a reasonable job of collating and sorting in UTF-8 locales. Perl now works with those. For portability and full control, Unicode::Collate is still recommended, but now you may not need to do anything special to get good-enough results, depending on your application. See "Category LC_COLLATE: Collation: Text Comparisons and Sorting" in perllocale
NUL charactersIn locales that have multi-level character weights, these are now ignored at the higher priority ones. There are still some gotchas in some strings, though. See "Collation of strings containing embedded NUL characters" in perllocale.
"{" characters in regular expression patterns are no longer permissibleYou have to now say something like "\{" or "[{]" to specify to match a LEFT CURLY BRACKET. This will allow future extensions to the language. This restriction is not enforced, nor are there current plans to enforce it, if the "{" is the first character in the pattern.
These have been deprecated since v5.16, with a deprecation message displayed starting in v5.22.
A variable name may no longer contain a literal control character under any circumstances. These previously were allowed in single-character names on ASCII platforms, but have been deprecated there since Perl v5.20. This affects things like $\cT, where \cT is a literal control (such as a NAK or NEGATIVE ACKNOWLEDGE character) in the source code.
/xxSpecifying two x characters to modify a regular expression pattern does everything that a single one does, but additionally TAB and SPACE characters within a bracketed character class are generally ignored and can be added to improve readability, like /[ ^ A-Z d-f p-x ]/xx. Details are at "/x and /xx" in perlre.
NBSP is no longer permissible in \N{...}The name of a character may no longer contain non-breaking spaces. It has been deprecated to do so since Perl v5.22.
CORE subroutines for hash and array functions callable via referenceThe hash and array functions in the CORE namespace--keys, each, values, push, pop, shift, unshift and splice--, can now be called with ampersand syntax (&CORE::keys(\%hash) and via reference (my $k = \&CORE::keys; $k->(\%hash)). Previously they could only be used when inlined.
A list of changes is at http://www.unicode.org/versions/Unicode9.0.0/. Modules that are shipped with core Perl but not maintained by p5p do not necessarily support Unicode 9.0. Unicode::Normalize does work on 9.0.
Note that some changed UCD database files in 9.0 stayed renamed to their shortened name in perl.
\p{script} uses the improved Script_Extensions propertyUnicode 6.0 introduced an improved form of the Script (sc) property, and called it Script_Extensions (scx). As of now, Perl uses this improved version when a property is specified as just \p{script}. The meaning of compound forms, like \p{sc=script} are unchanged. This should make programs be more accurate when determining if a character is used in a given script, but there is a slight chance of breakage for programs that very specifically needed the old behavior. See "Scripts" in perlunicode.
As an experimental feature, Perl now allows the referencing operator to come after my(), state(), our(), or local(). This syntax must be enabled with use feature 'declared_refs'. It is experimental, and will warn by default unless no warnings 'experimental::refaliasing' is in effect. It is intended mainly for use in assignments to references. For example:
use experimental 'refaliasing', 'declared_refs';
my \$a = \$b;See "Assigning to References" in perlref for slightly more detail.
Note that this still looks much worse than the perl6 bind operator: my $a := $b;
This adds a new modifier '~' to here-docs that tells the parser that it should look for /^\s*$DELIM\n/ as the closing delimiter.
These syntaxes are all supported:
<<~EOF;
<<~\EOF;
<<~'EOF';
<<~"EOF";
<<~`EOF`;
<<~ 'EOF';
<<~ "EOF";
<<~ `EOF`;The '~' modifier will strip, from each line in the here-doc, the same whitespace that appears before the delimiter.
Newlines will be copied as is, and lines that don't include the proper beginning whitespace will cause perl to croak.
For example:
if (1) {
print <<~EOF;
Hello there
EOF
}prints "Hello there\n" with no leading whitespace.
The old cperl -Dfortify_inc security feature was now also introduced by perl5 and renamed to -Ddefault_inc_excludes_dot.
Because the testing and make process for perl modules does not work well with . missing from @INC, cperl and perl5 still support the environment variable PERL_USE_UNSAFE_INC=1 which makes Perl behave as it previously did, returning . to @INC in all child processes.
Unlike utf8_hop(), utf8_hop_safe() won't navigate before the beginning or after the end of the supplied buffer.
@{^CAPTURE} exposes the capture buffers of the last match as an array. So $1 is ${^CAPTURE}[0].
%{^CAPTURE} is the equivalent to %+ (ie named captures)
%{^CAPTURE_ALL} is the equivalent to %- (ie all named captures).
cperl now sets the correct .pmc filename for __FILE__ and CopFILE, when it was loaded from it.
cperl also allows bypassing a .pmc if loaded explicitly via do and an absolute pathname.
This allows improved .pmc file caching of only selective parts of a module. Such as a method jit, which stores onlt some subs, but not the whole module in it's cache. Hence the Cache logic in the .pmc can now first load the parallel source .pm and then apply the .pmc optimizations. E.g. by loading a LLVM .bc file contents with only some subs.
The impact for existing code is low. If you loaded a .pmc via do "/abspath/module.pm" you need to add now a final "c" explictly: do "/abspath/module.pmc".
With perl5 upstream those two longstanding PMC bugs made it impossible to use a partial Byte- or JitCache. It also makes it possible to re-instate the old python-like timestamp logic which was removed for pugs 2006 with commit a91233bf4cf.
See [cperl #244]. cperl only.
SAFE_RX_CHECK_SUBSTR(rx)
SAFE_RX_ANCHORED_SUBSTR(rx)
SAFE_RX_ANCHORED_UTF8(rx)
SAFE_RX_FLOAT_SUBSTR(rx)
SAFE_RX_FLOAT_UTF8(rx)Other regex engines don't fill rx->substrs->data[], so it is unsafe to access it. Only allow ext/re and Perl_core_reg_engine. Currently only used in op_dump().
By reading malcrafted local Storable files or memory you could easily overwrite the local stack with controlled data. With bigger values you could cause an immediate exit, without backtrace or an exception being caught.
Another major stack-overflow fix is for [cpan #97526], limiting the maximal number of nested hash or arrays to 3000. Cpanel::JSON::XS has it at 512.
Note that p5p doesn't think that these are security issues. [perl #130635] (even if similar less severe attacks had a CVE and a metasploit module, which cperl detects).
cperl only so far. Uploaded to CPAN, but at this date still unauthorized.
On Unix systems, Perl treats any relative paths in the PATH environment variable as tainted when starting a new process. Previously, it was allowing a backslash to escape a colon (unlike the OS), consequently allowing relative paths to be considered safe if the PATH was set to something like /\:.. The check has been fixed to treat . as tainted in that example.
cperl as first dynamic scripting language follows the General Security Profile for identifiers in programming languages.
Moderately Restrictive: Allow Latin with other Recommended or Aspirational scripts except Cyrillic and Greek. Otherwise, the same as Highly Restrictive, i.e. allow :Japanese, :Korean and :Hanb.
"Some characters are not in modern customary use, and thus implementations may want to exclude them from identifiers. These include characters in historic and obsolete scripts, scripts used mostly liturgically, and regional scripts used only in very small communities or with very limited current usage. The set of characters in Table 4, Candidate Characters for Exclusion from Identifiers provides candidates of these."
cperl honors the TR31 Candidate Characters for Exclusion from Identifiers
I.e. You may still declare those scripts as valid, but they are not automatically allowed, similar to the need to declare mixed scripts.
use utf8;
my $ᭅ = 1; # \x{1b45} BALINESE LETTER KAF SASAK=> Invalid script Balinese in identifier ᭅ for U+1B45
use utf8 'Balinese';
my $ᭅ = 1; # \x{1b45} BALINESE LETTER KAF SASAK
print "ok";=>
okThe scripts listed at "Table 6, Aspirational Use Scripts": Canadian_Aboriginal, Miao, Mongolian, Tifinagh and Yi are included, i.e. need not to be declared.
With this restriction cperl fulfills the Moderately Restrictive level for identifiers by default. See http://www.unicode.org/reports/tr39/#General_Security_Profile and http://www.unicode.org/reports/tr36/#Security_Levels_and_Alerts.
Missing for more unicode security are warnings on single-, mixed and whole-script confusables, with a new utf8 warnings 'confusables' subcategory [cperl #265].
With special declarations of the used scripts or turning off no warnings 'utf8', you can weaken the restriction level to Minimally Restrictive.
All utf8 encoded names are checked for wellformed-ness.
chdir heap-buffer-overflow on the perl stackWhen called without argument it overwrote subsequent stack entries with the easily controllable result. [perl #129130]
This is merely a theoretical problem, improving on the previous sleep solution against hash floods. Distributed hashflood attacks could lead to memory exhaustion and denial of service in threaded servers, which would bypass the original FAIL_DELAY-like intrusion detection and mitigation.
First sleep, but if >128 concurrent attacks are detected, exit hard. Use a global hash_slowdos counter. Note that this is also triggered by a 128*8*128 hash collision single source attack (=131072). This is still better, faster and smaller than the java solution to convert the linked list to a tree. We log the attackers and can block them. [cperl #246]. cperl only.
@{ \327 \n } buffer overflowsFixed @{ \327 \n } tokenizer failures and heap buffer overflows in sv_vcatpvfn_flags() with wrong tracking of PL_linestr, the currently parsed line buffer. This can easily lead to security relevant exploits.
eval "q" . chr(overlarge) stack overflowIn eval "q" . chr(100000000064) generating the error message Can't find string terminator "XXX"' was overrunning a local buffer designed to hold a single utf8 char, since it wasn't allowing for the \0 at the end.
If the collisions for a hash key lookup exceeds 128 tries (i.e. a linear search in a linked list), this qualifies as a malicious hash DoS (Denial of Service) attack. Generally maximal 8-10 collisions appear in normal hash table usage. Every 8th such hash flood attack performs a sleep(2) to limit the impact.
Detect and protect against it, also call the new warn_security("Hash flood").
This security scheme is much easier and faster than trying to hide the random hash seed with randomized iterators and collisions lists, which cperl doesn't use.
See "New Diagnostics".
In order to avoid TR39 confusable security hacks, we add the following unicode rules for identifiers and literals with mixed script properties:
The 'Common', 'Latin' and 'Inherited' scripts are always allowed and don't need to be declared.
The first non-default unicode script for an identifier is the only allowed one. This qualifies as single-script. More scripts lead to parsers errors.
Additional unicode scripts can and should be declared via use utf8 'Greek', 'script-name2'.... This allows mixed scripts in identifiers. This can be scoped in blocks.
To fulfill the Moderately Restrictive Level for the Unicode General Security Profile you may not mix Greek with Cyrillic identifier characters in the same program.
See http://www.unicode.org/reports/tr39/#Mixed_Script_Detection and [cperl #229]
This holds for all identifiers (i.e. all names: package, gv, sub, variables) and literal numbers.
Currently there exist 131 scripts, see "Valid scripts" in utf8.
All stored utf8 names, identifiers and literals are parsed and stored as normalized NFC unicode, which prevents from various TR39 and TR36 unicode confusable and spoofing security problems.
However, dynamically created symbols via string refs are not normalized. ${"$decomposed"} stays decomposed.
Note that even perl6 stores different names for confusables, which match each other due to their NFG rules on their string matchers. perl5 matches strictly binary, which leads to confusable and spoofing security problems.
See [cperl #228], http://www.unicode.org/reports/tr36/, http://www.unicode.org/reports/tr39, http://www.unicode.org/reports/tr31/ and the Python 3 discussion 2007 on PEP 3131 https://docs.python.org/3/reference/lexical_analysis.html#identifiers.
Python 3 normalizes to NFKC (Compatibility Decomposition, followed by Canonical Composition), cperl uses both canonical transformations. See http://unicode.org/reports/tr15/#Norm_Forms for the difference. Basically NFKC transforms to shorter ligatures. NFC is recommended by TR15.
Fallback to the secure behvaiour as before v5.16 and strip symbol names of everything after the first \0 character. This protects from creating binary symbols as with no strict 'refs'; ${"a\0\hidden"}, which were especially problematic for package names, which were mapped 1:1 to filenames. With the default warning 'security' in effect, a warning is produced by the "warn_security" in perlapi API, same as for unsafe syscalls since 5.20.
See "Invalid \0 character in string for SYMBOL: %s" in perldiag and [cperl #233].
cperl5.22.2 added a restraint to expose the internal hash secret seed via the environment variable PERL_HASH_SEED_DEBUG=1 to be hidden in taint mode. See [cperl #114] and "Core Enhancements" in perl5222cdelta.
PERL_HASH_SEED_DEBUG=1 cperl5.22.2 -e1 =>
HASH_FUNCTION = FNV1A HASH_SEED = 0xecfb00eb PERTURB_KEYS = 0 (TOP)
PERL_HASH_SEED_DEBUG=1 cperl5.22.2 -t -e1 => emptyBut unfortunately not many perl services are actually protected with -t, even if cperl fixed taint mode to be actually secure. The seed exposure is only needed for a debugging perl, and actually is security relevant.
So PERL_HASH_SEED_DEBUG=1 will now hide the seed value in non-DEBUGGING builds.
PERL_HASH_SEED_DEBUG=1 cperl5.25.2 -e1 =>
HASH_FUNCTION = FNV1A HASH_SEED = <hidden> PERTURB_KEYS = 0 (TOP)Note that the seed is still trivially exposable via other means if a local script can be executed, as the seed value is readable from a fixed memory offset via unpack "P". That's why cperl fixed hash table security via proper means in the collision resolution, not via a slow hash function, and not via order hiding as perl5 believes in.
More discussion at https://github.com/google/highwayhash/issues/28 and https://github.com/google/highwayhash/issues/29.
Detection of the destructive attack against Movable-Type, the third vector only, which tries to delete mt-config.cgi was added to was added to Storable 3.01c.
Calls warn_security("Movable-Type CVE-2015-1592 Storable metasploit attack"), but does not protect against it.
Detect the metasploit payload unix/reverse_perl and some existing variants. This is just a dumb match at startup against existing exploits in the wild, but not future variants. Calls warn_security("metasploit reverse/bind shell payload"), but do not protect against it. This warning is thrown even without -w.
Also detects the CVE-2012-1823 reverse/bind shell payload, which is widely exploited too. The security warning is called "CVE-2012-1823 reverse/bind shell payload".
With a warnings 'syscalls' violation, i.e. detecting \0 in arguments to C API syscalls, the new 'security' warnings category overrides the 'syscalls' category. I.e. the warning is produced by the "warn_security" in perlapi API, and to turn it off, you have to turn off both categories.
See the new perldeprecation pod.
Many old deprecations got now a fixed final date, but several perl5 deprecations were undeprecated in cperl and rather fixed. (as in previous cperl releases.)
Those macros clash on darwin XTools with the system iostream _OutputIterator methods. We need to use the fullname Perl_do_open and Perl_do_close functions whenever perl needs to be embedded into C++ projects.
With the system C++ compiler on darwin do_open, do_close are now undefined. See [cperl #227]
Made something like sub foo'bar; a syntax error. ' is not replaced by :: anymore when used as package seperator. This was deprecated 10 years ago.
cperl fixed the "c2ph" core utility using this last remaining perl4'ism, and removed the isn't method from Test::More. In a later versions ' can be reenabled as proper IDContinue character for identifiers, e.g. for Test::More isn't.
See [cperl #217].
Attribute "locked" is deprecated, and will disappear in Perl 5.28
Attribute "unique" is deprecated, and will disappear in Perl 5.28
Constants from lexical variables potentially modified elsewhere are deprecated. This will not be allowed in Perl 5.32
Deprecated use of my() in false conditional. This will be a fatal error in Perl 5.30
File::Glob::glob() will disappear in perl 5.30. Use File::Glob::bsd_glob() instead.
%s() is deprecated on :utf8 handles. This will be a fatal error in Perl 5.30
$* is no longer supported. Its use will be fatal in Perl 5.30
$* is no longer supported. Its use will be fatal in Perl 5.30
Opening dirhandle %s also as a file. This will be a fatal error in Perl 5.28
Opening filehandle %s also as a directory. This will be a fatal error in Perl 5.28
Setting $/ to a reference to %s as a form of slurp is deprecated, treating as undef. This will be fatal in Perl 5.28
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/%s/
Unknown charname '' is deprecated. Its use will be fatal in Perl 5.28
Use of bare << to mean <<"" is deprecated. Its use will be fatal in Perl 5.28
Use of code point 0x%s is deprecated; the permissible max is 0x%s. This will be fatal in Perl 5.28
Use of comma-less variable list is deprecated. Its use will be fatal in Perl 5.28
Use of inherited AUTOLOAD for non-method %s() is deprecated. This will be fatal in Perl 5.28
Use of strings with code points over 0xFF as arguments to %s operator is deprecated. This will be a fatal error in Perl 5.28
In order for Perl to eventually allow string delimiters to be Unicode grapheme clusters (which look like a single character, but may be a sequence of several ones), we stop allowing a single char delimiter that isn't a grapheme by itself. These are unlikely to exist in actual code, as they would typically display as attached to the character in front of them.
E.g. qr ̂foobar̂; is now an error, it is only deprecated with v5.25.9 upstream and will be illegal in perl5 v5.30. cperl only.
perl5.25.3 started allowing state variables in loops. cperl still disallows them.
perl5.25.3 -E'use feature "declared_refs","refaliasing";
for state \$x (\$y) { print $x }'
=> warnings: Declaring references is experimental at -e line 1.
Aliasing via reference is experimental at -e line 1.
cperl5.25.3 -E'use feature "declared_refs","refaliasing";
for state \$x (\$y) { print $x }'
=> error: Missing $ on loop variable at -e line 1.and without declared_refs:
perl5.25.3 -E'for state $x ($y) { print $x }'
cperl5.25.3 -E'for state $x ($y) { print $x }'
=> error: Missing $ on loop variable at -e line 1.scalar(%hash) return value changedThe value returned for scalar(%hash) will no longer show information about the buckets allocated in the hash. It will simply return the count of used keys. It is thus equivalent to 0+keys(%hash).
A form of backwards compatibility is provided via Hash::Util::bucket_ratio() which provides the same behavior as scalar(%hash) provided prior to Perl 5.25.
keys returned from an lvalue subroutinekeys returned from an lvalue subroutine can no longer be assigned to in list context.
sub foo : lvalue { keys(%INC) }
(foo) = 3; # death
sub bar : lvalue { keys(@_) }
(bar) = 3; # also an errorThis makes the lvalue sub case consistent with (keys %hash) = ... and (keys @_) = ..., which are also errors. [perl #128187]
Seperate an unlikely codepath in scalar assignments (ASSIGN_CV_TO_GV) to another function, helping the CPU instruction cache. 10% faster on Intel.
Ensure that the lexer always sees large enough buffers to do fast wordwise memcmp comparisons, esp. with constant lengths.
The initial size of empty arrays went in cperl from 4 to 2, AvMAX = 1. Array speed is 2-15% faster on perlbench, overall speed the fastest of all so far. Memory win: <0.1%
length in boolean context without get magic doesn't need to calculate the utf8 length, it only needs to check if SvCUR field is empty. And it doesn't need to allocate a new IV for the result, just use the existing sv_yes or sv_no. Analog to [cperl #245] for ref. cperl only.
ref in boolean context doesn't need to allocate a string. 2-3x faster. See [cperl #245] and [perl #78288] cperl only. in perl5 announced for 5.28.
Reading from a file line-by-line with readline() or <> should now typically be faster due to a better implementation of the code that searches for the next newline character.
$ref1 = $ref2 has been optimized.e.g.
(..., @a) = (...);
(..., %h) = (...);especially when the RHS is empty.
Note that perl5 hash assignment is still inferior to cperl hash assignment.
Reduce the number of odd special cases for the SvSCREAM flag.
Avoid sv_catpvn() in do_vop() when unneeded.
Enhancements in Regex concat COW implementation.
The internal op implementing the split builtin has been simplified and sped up. Firstly, it no longer requires a subsidiary internal pushre op to do its work. Secondly, code of the form my @x = split(...) is now optimised in the same way as @x = split(...), and is therefore a few percent faster. This required B::* compiler changes.
Bareword constant strings are now permitted to take part in constant folding. They were originally exempted from constant folding in August 1999, during the development of Perl 5.6, to ensure that use strict "subs" would still apply to bareword constants. That has now been accomplished a different way, so barewords, like other constants, now gain the performance benefits of constant folding.
This also means that void-context warnings on constant expressions of barewords now report the folded constant operand, rather than the operation; this matches the behaviour for non-bareword constants.
Most NULL ops are now removed in the peephole optimizer. Check for #if defined(PERL_REMOVE_OP_NULL) in your XS module if you hardcoded any NULL-sensitive op-tree structure. See how many with -Dk.
-DPERL_FAKE_SIGNATURE is now default, making most function calls 2x faster. See "fake_signatures"
The new compiler option support allows generation of much faster code. I.e. clang-4.0 with -flto or zapcc produce ~20% faster code.
for loops got several enhancements:
new special iter_ary for (@ary) and iter_lazyiv for (0..9) ops to avoid a run-time switch in the generic iter op.
more aelem_u optimizations, less run-time out of bounds checks for shaped arrays in loops. E.g. in my @a[5]; $a[$_] for (0..4); the compilers knows that the max index for @a will be 4, which is within the allowed shape of @a.
The multideref OP has a new MDEREF_INDEX_uoob flag. This is used for unchecked out-of-bounds checks for arrays, to use the previous AvSHAPED array optimizations (aelem_u, aelemfast_lex_u) or loop out-of-bounds elimination with multideref OPs also. Such multideref ops appear pretty often even with single indices. E.g. in my @b=(0..4); for (0..$#b) { $b[$_] = 0; } $b[$_] is converted to a multideref, which previously was not optimized.
Those optimized indices are marked with a new " _u" suffix in the dumped multideref stringification.
MDEREF_MASK changed to 0x10F, the MDEREF_SHIFT size from 7 to 8. The shift can also use faster intrinsics now.
The loop out-of-bounds elimination was fixed for simple lexical indices (e.g. for my $i (0..$#a){ $a[$i] }, which leads now to more aelem_u ops and subsequent mderef_u optimizations also.
The new strEQc/strNEc macros are used instead of strEQ(s,"constant"). This enables word-wise comparison via memcpy, in opposite of byte-wise comparisons via strcmp with already known sizes. This is a 10% performance improvement under most optimization levels.
Use more strEQc, strNEc macros, when safe to use, i.e. the left buffer is big enough, now with Address Sanitizer fallbacks.
The new fast buffer comparison macros strEQc and strNEc compare a full string including the final \0, memEQc and memNEc just the start of a buffer, with constants strings. Note that valgrind and Address Sanitizer will complain about out of range access of the left side of the buffer. To access these buffers however is safe and will not lead to SIGBUS on stricter platforms. To prevent valgrind from warning on this, you may want to define -DVALGRIND, which uses a safe and slower fallback macro.
Make all padnames not UTF8 per default, only the ones which are really UTF8. See "Internal Changes" and [cperl #208]
Improvements when reading from arrays have been imported from perl5. av_fetch() uses less branches reading from the end (negative indices), and a branch checking for freed @_ elements has been removed,
Extract hv_common_magical() to a seperate function. Extracts uncommon magical code in hot code to an extra static function to help keep the icache smaller. Only in rare cases this branch is taken. I.e filling ENV at startup, or using tied hashes.
Measured 2-15% faster with normal scripts, not using tied hashes.
aassign: pre-allocate needed hash size with aassign, similar to arrays, avoiding run-time hash splits. e.g. my %h = (.. = .., .. => ..)>
This version is 30% faster overall in the Mail::SpamAssassin testsuite than cperl-5.25.0.
Pre-extend internal hashes and stashes to avoid unnecessary boot-time hash splits. %warnings::, %Config::, %utf8::, %version::.
Added new get_svs, get_avs, get_hvs macros, and accompanied get_[ash]vn_flags API functions, to omit the run-time strlen(name) for constant names. (#191)
Better 09_roundtrip.t tests.
Handle tarballs compressed with pbzip2 (RT #119262)
Add missing strict/warnings pragma to Constants.pm
Check for gzip/bzip2 before round tripping gz/bz2 files in tests
Use op_class API
Allow a 2nd optional CV argument for B::OP::aux_list, fixing B::Deparse and thereby Data::Dumper and Test2 is_deeply.
use the new get_svs, get_avs, get_hvs macros.
Fixes for PERL_OP_PARENT: moresib, sibling, parent.
Fix hints/522_patched.pl dependency on C.so [cpan #120161]
PUSHRE replaced by SPLIT, no xpad_cop_seq, SVpbm_VALID
Improved dl_module_to_sofile without 2nd arg
Fixed IsCOW savepvn, store the last cowrefcnt.
Fixed wrong savepvn length, failing with asan.
Optimized mro_isa_changed_in initialization.
Better CopFILE_set, Fixup arenasize refcnt. Delay cvref to init2, properly set a SvRV to a XS sub. Optimize constpv for CvFILE (less constants to merge for gcc). Improve NV precision by one digit. Fix to compile in utf8_heavy.pl, abstract and set %INC. Fix generation of @B::C::Config::deps on Windows. Fix !C99 precedence bug (e.g. MSVC). Minor refactor to simplify save_hek. Use the new get_svs, get_avs, get_hvs macros. perlcc add --debug|-D Improve endav XSUB bump Abstract RITER_T and HVMAX_T for the various sizes, compat HEK_STATIC Defer REGCOMP for \P{} properties Change $sv->EXTFLAGS to compflags since 5.22 for CALLREGCOMP(). Turn off MGf_REFCOUNTED. global-buffer-overflow with dynamic COW strings, wrong savepvn args.
Support 5.25.6 split optimization
Update deprecation message
Handle chunk errors phrases
Need Fix for Makefile.PL depending on . in @INC RT #120084
Comment out unused variables & remove C++-ism. RT #120272
And merge cpan/Parse-CPAN-Meta into it. cpan/Parse-CPAN-Meta is gone.
Parse-CPAN-Meta security: set $YAML::XS::DisableCode, $YAML::XS::DisableBlessed.
Add support for all known YAML and JSON modules: *::Syck, JSON::MaybeXS, Mojo::JSON. But JSON::Any is broken.
fixed UTF-8 issues, passes now all Test-CPAN-Meta tests.
with full cperl support. reapply most of our patches. skip cperl builtin prereqs.
Moved from cpan to dist. [cperl #154].
- Fix need() overallocation (#84 Matthew Horsfall) and missing need() calls.
- Fix decode_prefix offset when the string was re-allocated. rather return the relative offset not the pointer to the old start.
- Fixes for g++-6, stricter -fpermissive and -Wc++11-compat.
- Added tests for ill-formed utf8 sequences from Encode.
- modfl() mingw 4.0 runtime bug [perl #125924]
- Tested with the comprehensive JSON decode spectests from http://seriot.ch/parsing_json.html. Not added to core. #72
- decode with BOM: UTF-8, UTF-16, or UTF-32.
- fixed detection of final \0 as illegal non-whitespace garbage. Fixes spectest 'n_number_then_00'. #72
- warn with unicode noncharacters as in core when not in relaxed mode. #74
- fail decode of non-unicode raw characters above U+10FFFF when not in relaxed mode.
- New stringify_infnan(3) infnan_mode.
- Fix inf/nan detection on HP-UX and others.
- Use faster strEQc macros.
- Prefer memEQ for systems without memcmp, to use bcmp there.
- Add more expect_false() to inf/nan branches.
- Fix av and hv length types: protect from security sensitive overflows, add HVMAX_T and RITER_T
- Add new "Hash key too large" error. perl5 silently truncates it, but we prefer errors.
protect sv in END during global destruction, esp. with B::C. fixes for missing . in @INC (cperl or -Dfortify_inc).
Fix -Wc++11-compat warnings
Fix -Wc++11-compat warnings
strEQc improvements
fix correct indentation for utf-8 key hash elements, [perl #128524].
Fix -Wc++11-compat warnings, and various minor issues.
use the new get_svs, get_avs, get_hvs macros. The flags where harmonized, missing names were added, most fields are now print in natural order as in the struct.
Fix -Wc++11-compat warnings
prevented shasum from possibly running malicious code, remove '.' from @INC before module loading RT #116513, namespace cleanup (RT #105371 and #105372), minor code and documentation tweaks
unused arg warnings RT #107642
The other 2 fixes were already in cperl, plus a fix for reproducible builds.
Fixed dl_findfile refcounts, "panic: attempt to copy freed scalar" errors.
Fixed build dependency for dlboot.c. No excessive rebuilds anymore.
no mathoms: call_sv instead of call_pv, get_cvs where available.
use the new get_svs, get_avs, get_hvs macros.
various upstream fixes. plus g++-6 -fpermissive and -Wc++11-compat fixes. our local make -s silent patches and various others are now all upstream.
Exporter remained unchanged. But CORE support for the "used only once" warnings has been to restricted to the four magic names "EXPORT", "EXPORT_OK", "EXPORT_FAIL" and "EXPORT_TAGS". Other names starting with "EXPORT" will now throw the "used only once" warning as all other symbols.
Fix ProxySubs to generate code also valid for 5.6 - 5.14. Add testcases for all ProxySubs options.
Fix -Wc++11-compat warnings in generated const-xs.inc code.
fix \Q$archname\E in t/basic.t
skip cperl builtin prereqs.
ExtUtils::Liblist::Kid:
one more darwin fix for the wrong no library found warning for symlinked darwin libSystem.dylib libraries.
support make -s => PERL_INSTALL_QUIET
XS_EXTERNAL does now extern "C"
Fix visibility declaration of XS_EXTERNAL for -flto and -fvisibility=hidden.
Revise documentation of eval and evalbytes
use the new get_svs, get_avs, get_hvs macros.
* Set a cleaned env when running git clone * Changed git repository link in tests * Removed consistently failing httpbin.org tests * Require Module::Load::Conditional 0.66 * Fix FTP tests for ipv6
Deprecated File::Glob::glob()
use the new get_svs, get_avs, get_hvs macros.
added 2 more pure tests
Fixes for . in @INC
added filter-util.pl to t/
fixed INSTALLDIRS back to site since 5.12 [gh #2]
Changed pod NAME to follow convention.
* RT #114999 fix :number * RT #113748 fix VersionMessage ignores -output argument * RT #39052 sanify gnu_getopt
Many fixes und improvements
fix for PERL_GLOBAL_STRUCT_PRIVATE Fix -Wc++11-compat warnings
ISA fixes for c3 [perl #120239
protect sv in END during global destruction, esp. with B::C. fixes for missing . in @INC (cperl or -Dfortify_inc).
From https://github.com/atoomic/IO-Socket-IP/:
- Support setting custom socket options with new Sockopts constructor parameter
- Restore blocking mode after ->connect errors RT #112334
- Keep demand-load Carp patch. https://jira.cpanel.net/browse/CPANEL-4359
set $Module::Load::Conditional::FORCE_SAFE_INC = 1
Fixed true/false redefinition warnings.
Added Czech republic aliases back in. Lot of new codes and code changes.
Made many private API calls public. See Locale::Codes::Changes
Fix optional runtime load for CVE-2016-1238
Add blacklist and whitelist support, with [perl #127923 priority. See "BRACKET NOTATION SECURITY" in Locale::Maketext
Merged CPAN updates with stricter tests and . in @INC fixes. https://github.com/rurban/Math-BigInt
Updated from CPAN:
2 new tests files from Math-BigInt.
Math::BigInt::FastCalc is now a subclass of Math::BigInt::Calc, so remove aliases like *Math::BigInt::FastCalc::_xxx = \&Math::BigInt::Calc::_xxx.
Use OO-calls rather than function calls. (i.e slower but overridable)
Updated from CPAN: No functional changes, and the few actual changes in the test lib were for the worse. The bigfltpm.inc tests are still broken and skipped, however I improved it.
Fix unconditional @INC localisation, Add FORCE_SAFE_INC option to fix CVE-2016-1238.
- Fix file operation in tests for VMS
- use a more strict matching heuristic when attempting to infer the "primary" module name in a parsed .pm file
- only report "main" as the module name if code was seen outside another namespace, fixing bad results for pod files (RT#107525)
Stabilized test, geocities.com is down.
Return the port num as 5th return value with ack (jfraire).
Todo 010_pingecho.t on EBCDIC and os390. Skip udp ping tests on more platforms: hpux, irix, aix (all from perl5)
Fixed missing _unpack_sockaddr_in family, which took AF_INET6 for a AF_INET addr in t/500_ping_icmp.t and t/500_ping_icmp_ttl.t. Use now a proper default. Detected by the new gitlab ci.
Fixed _pack_sockaddr_in for a proper 2nd argument type, hash or packed address.
Improved 500_ping_icmp.t to try sudo -n for tests requiring root, plus adding -n fir fixing [RT #118451]. Relaxed more tests failing with firewalled icmp on localhost. [RT #118441]
Fixed ping_external argument type, either packed ip or hostname. [RT #113825]
Fixed wrong skip message in t/020_external.t
Doc and meta changes only.
- Remove . from @INC when loading optional modules. [Tony Cook, Perl RT#127834, CVE-2016-1238]
- Increased minimum required version of IO::Socket::IP to 0.25 to hopefully stop t/pop3_ipv6.t hanging. [CPAN RT#104545]
- Debug output now includes decoded (from base64) negotiation for SASL. [Philip Prindeville, PR#27]
- plus the suse utf8 fixes for Net::Cmd, see 5bd7010cb and our darwin performance fix for hostname.
New iter_ary and iter_lazyiv ops.
Add avhvswitch op
improved t/parent-pmc.t, excluded new xt tests
pod: Suggest to use strict :encoding(UTF-8) PerlIO layer over not strict :encoding(utf8) For data exchange it is better to use strict UTF-8 encoding and not perl's utf8.
Add security note to File::Spec::no_upwards [RT #123754
use the new get_svs, get_avs, get_hvs macros.
Omit wrong if (gv) check, detected by coverity.
Added msys
fix cache races with parallel tests. add the PID to the temp. cache file See [RT #118416].
Removed deprecated --libpods option.
Add the t/data/snippets tests.
Use Pod::Simple's logic to determine the native code points for NO BREAK SPACE and SOFT HYPHEN instead of hard-coding the ASCII values. Hopefully fixes the case of mysterious disappearing open brackets on EBCDIC systems. (#118240)
Many Pod::Man bugfixes and new tests, see https://metacpan.org/changes/distribution/podlators
Fix broken test on Windows and FreeBSD (RT#116551) Fix CVE-2016-1238 by temporarily removing '.' from @INC in world writable directories. Fix =head3 appearing in some perlfunc lookups AmigaOS patches (RT#106798) (RT#110368) Fall back to an English perlfunc if translation doesn't exist (RT#104695) FreeBSD has mandoc too, with UTF-8 support. -U now documented and implied with -F (RT#87837) Fixes less -R flags insertion on Windows [perl #130759]
Updated from CPAN 3.35: Turn off utf8 warnings when trying to see if a file is UTF-8 or not
Merged with our cperl signature modernizations, tracked at https://github.com/rurban/pod-simple/tree/cperl.
Moved from cpan to dist. [cperl #154].
changed speed_t to unsigned long. cannot be negative.
Fix -Wc++11-compat warnings
Several defects in making its symbols exportable. [perl #127821]
The POSIX::tmpnam() interface has been removed, see "POSIX::tmpnam() has been removed" in perl5251delta.
Trying to import POSIX subs that have no real implementations (like POSIX::atend()) now fails at import time, instead of waiting until runtime.
use the new get_svs, get_avs, get_hvs macros.
Documentation only
Bumped version because upstream is still years behind: lexical $_ support, binary names, various other fixes.
Improved taint test.
sum/min/max need to call SvGETMAGIC
set_subname memory fix by @bluhm from Sub::Name 0.20 RT #117072
Fixes for older perls, esp. lexical $_ support.
Reinstate the &DB::sub setter, but no UTF8 support yet.
Fix -Wc++11-compat warnings
Merge cpan 2.024 with our 2.021_02, plus fix some problems in their new code.
Fixes for OpenBSD: Probe for netinet/in_systm.h Removed i_netinet6_in6 probe. This was never used due to a typo. It cannot be used due to RFC 2553.
From https://github.com/rurban/Storable
Added some I32_MAX checks for tainted integers.
Fixed 3 null ptr dereferences leading to segfaults. [perl #130098]
Fixed some important security bugs with reading from Storable files or memory, directly controlling the stack (not the perl stack). See "Storable stack overflow or exit".
Another stack-overflow fix is for [cpan #97526], limiting the maximal number of nested hash or arrays to a computed number, ~3000-8000. Cpanel::JSON::XS has it at 512.
Fixed up early huge buffer and index support from 3.00c, which was failing with wrong malloc errors due to silently overwrap >2GB. t/huge.t works now correctly. Note that 2 cases are not relevant since v5.25.1c/v5.24.1c anymore as with these releases we limit the maximum number of elements for hashes and arrays, and fail with "Too many elements" before. Bumped up PERL_TEST_MEMORY requirements to 8 and 16 for arrays and hashes. In reality the VMM subsystem will kill the process on perl5 before. $a[9223372036854]=0 or %a=(0..4294967296) are easy ways to DoS a perl5 system. Only cperl is safe.
Skip or croak when reading or writing 64bit large objects on 32bit systems.
Fixed wrong recursion depth error with large arrays containing another array. Probe for valid max. stack sizes, and added 2 APIs. [cperl #257]
Update documentation from the CPAN version.
CVE-2016-1238: avoid loading optional modules from default . (Tony Cook). Patch rewrote to no longer depend upon @INC. See https://metacpan.org/changes/distribution/Sys-Syslog
Kept our smoker logic in t/syslog.t, for slow darwin systems, the suse patch and disabled the lexical filehandle patch.
Add aliases ansi16 through ansi255 and on_ansi16 through on_ansi255 (plus the corresponding constants) for the grey and rgb colors so that one can refer to all of the 256 ANSI colors with consistent names. These are aliases; the colors returned by uncolor will still use the grey and rgb names. (#118267)
ReadKey.pm renamed to ReadKey_pm.PL, expand blockoptions specific variants already at installation, no load-time eval, demand-load Carp, remove unneeded AutoLoader, harmonize formatting.
patch: use faster StructCopy and fixup the XS.
Keeping our cperl and XSConfig fixes. See https://github.com/rurban/Test-Harness/tree/cperl-rebased
Add doc and support for optional subtest @args.
Moved from cpan to dist. [cperl #154].
Removed the deprecated isn't method, using the ' package seperator.
Remove extra terminating semicolon. Clean up temporary directories after testing.
Calling any of the dequeue methods with COUNT greater than a queue's limit will generate an error.
But still have to keep our test fixes for . in @INC
Added down_timed method.
ifdef clang
Documentation
Less runtime memory: demand-load Carp, Config.
Fix -Wc++11-compat warnings Keep our better C++ fixes Keep our t/usleep.t, t/alarm.t, t/utime.t fixes. Keep our do_openn improvements in typemap.
from upstream:
- El Capitan compatibility - use CLOCK_REALTIME for clock_nanosleep scan - include file consistency in scans - use clockid_t consistently - use hv_fetchs() - scan for clockid_t (needed for macos Sierra) - darwin lacks clockid_t [cpan #129789]
More Darwin thread fixes for clock_gettime, Sierra support, test improvements, skip the t/utime.t on ext2/ext3
Many new locales. Some major fixes.
Merge latest version with the '_' lyon concensus with the cperl extension of the optional final 'c' suffix. Extend version::regex for cperl. Now also parse the 'c' natively.
merged with upstream libyaml 0.1.7 avoid duplicate checks against NULL fix libyaml clang -Wlogical-op warnings fix libyaml clang -Wlogical-not-parentheses warnings
fixed encoding issues: fixed wrong $YAML::XS::Encoding and $YAML::XS::LineBreak comparison logic. fixed utf8 input as handled as UTF8, non-utf8 honors $YAML::XS::Encoding.
fixed -Wunused value warnings
merged with upstream YAML-LibYAML, implemented $DisableBlessed (security).
use the new get_svs, get_avs, get_hvs macros.
coverity detected invalid branch: if (!module) is always false. we need to check if (!modlibname) instead.
Compared 8-bit scalar data according to the current locale.
Deprecated with 5.003_06. Its functionality was integrated into the Perl core language in the release 5.003_06.
See perllocale for further information.
This file documents all upcoming deprecations, and some of the deprecations which already have been removed. The purpose of this documentation is two-fold: document what will disappear, and by which version, and serve as a guide for people dealing with code which has features that no longer work after an upgrade of their perl.
Added a section on calling methods using their fully qualified names.
Do not discourage manual @ISA.
Clarify behavior single quote regexps.
Some locales aren't compatible with Perl. Note the potential bad consequences of using them.
Deprecations are to be marked with a D. "%s() is deprecated on :utf8 handles" use a deprecation message, and as such, such be marked "(D deprecated)" and not "(W deprecated)".
Describe which system have Dtrace. Changes in cperl. New examples.
add pTHX_ to magic method examples.
Document @ISA. Was documented other places, not not in perlvar.
Tidy the document.
Mention Moo more.
Describe the "CPERL" in perlhack development model, with always updated branches, git rerere and cp-rb.
Document Tab VS Space.
Several minor enhancements to the documentation.
perlinterp has been expanded to give a more detailed example of how to hunt around in the parser for how a given operator is handled.
Removed redundant dSP from an example.
Updated documentation of scalar(%hash). See "scalar(%hash) return value changed" above.
Use of single character variables, with the variable name a non printable character in the range \x80-\xFF is no longer allowed. Update the docs to reflect this.
Updated documentation of scalar(%hash). See "scalar(%hash) return value changed" above.
Described the Moderately Restrictive level for unicode identifiers.
Added new cperl v5.25.2c restrictions for "Identifier parsing" in perldata: No undeclared mixed scripts, normalization to NFC and no binary symbols.
Documented new feature: See "Declaring a reference to a variable" above.
Document removed experiments: "Lexical Subroutines" in perlsub is now default. Array and hash container functions accept references removed.
Defined on aggregates is no longer allowed. Perlfunc was still reporting it as deprecated, and that it will be deleted in the future.
Clarified documentation of seek(), tell() and sysseek(). [perl #128607]
Removed obsolete documentation of study().
Documented change to \p{script} to now use the improved Script_Extensions property. See "Use of \p{script} uses the improved Script_Extensions property" above.
Updated the text to correspond with changes in Unicode UTS#18, concerning regular expressions, and Perl compatibility with what it says.
Removed obsolete documentation of ${^ENCODING}. See "${^ENCODING} has been removed" above.
All references to Usenet have been removed.
All references to Usenet have been removed.
Document NUL collation handling.
All references to Usenet have been removed.
Updated the mirror list.
All references to Usenet have been removed.
All references to Usenet have been removed.
Added documentation for all op functions.
Describe the "Taint mode" in perlsec differences (hash keys, use re 'taint'), added a "use warnings 'security'" in perlsec paragraph.
For hashes describe the different PERL_PERTURB_TOP strategy regarding Bucket Order Perturbance, add more text to "Alternative Hash Functions" in perlsec and add a new "cperl hash security" in perlsec paragraph.
The following additions or changes have been made to diagnostic output, including warnings and fatal error messages. For the complete list of diagnostic messages, see perldiag.
Added a new warnings category security which is default ON, using a special message.
A "SECURITY: " prefix, and as suffix the username, REMOTE_ADDR, full pathname to implement a service similar to fail2ban. Bypass $SIG{__WARN__} handlers. Prints to STDERR and if available to syslog.
A new -Dmv debugging mode for verbose arena memory debugging was added, similar to -Dm and env PERL_MEM_LOG=s.
The Too many elements error is now triggered when accessing or extending an out of bounds array index or trying to insert too many hash keys. This is to prevent from silent hash or array overflows. Previously extending a hash beyond it's capable size was silently ignored, leading to performance degradation with overly high fill factors and extending an array failed only on memory exhaustion, but the signed index led to an index overflow between I32 and U32, resp. I64 and U64.
Even worse, accessing overflown unsigned array indices would silently access the signed counterpart, indices at the end.
Note that the out of bound error message with shaped arrays is different.
The Panic: hash key too long error is now thrown with overlarge hash keys in every hv_common access and in Cpanel::JSON::XS. perl5 still silently ignores those failures, and truncates the keys.
Many more similar panic: (file|keyword|mro|stash)? name too long errors were added to the parser, compiler and runtime to protect from overlong names (> I32_MAX, 2147483647, 2GB), or counts.
Invalid for range iterator (%d .. %d)
(F) A range with constant integers as a for loop cannot be reversed.
We check now at compile-time that a range with for loops with constant integers is incremental, the 2nd number must be higher. We don't support reverse loops with ranges, i.e. for (9..0) is invalid, but for (0..$#ary) with $#ary being -1 is valid.
Reverse constant strings ranges are still valid and lead to an empty loop. i.e. for ('z'..'a') is currently valid.
Using the empty pattern (which re-executes the last successfully-matched pattern) inside a code block in another regex, as in /(?{ s!!new! })/, has always previously yielded a segfault. It now produces an error: Infinite recursion via empty pattern, (was Use of the empty pattern inside of a regex code block is forbidden).
The experimental declared_refs feature is not enabled
(F) To declare references to variables, as in my \%x, you must first enable the feature:
no warnings "experimental::declared_refs";
use feature "declared_refs";Unescaped left braces are already illegal in some contexts in regular expression patterns, but, due to an oversight, no deprecation warning was raised in other contexts where they are intended to become illegal. This warning is now raised in these contexts.
Malformed UTF-8 character in compose (empty string)
Malformed UTF-8 character in decompose (empty string)
Malformed UTF-8 character in reorder (empty string)
New unicode identifier normalization errors.
The new S security warning "Hash flood" was added. See "Protect and warn on hash flood DoS".
The new S security warnings "metasploit reverse/bind shell payload" and "CVE-2012-1823 reverse/bind shell payload" were added, detecting the existing metasploit/libxploit and phpcgi CVE-2012-1823 reverse and bind shells. See "Warn on metasploit reverse shells"
Declaring references is experimental
(S experimental::declared_refs) This warning is emitted if you use a reference constructor on the right-hand side of my(), state(), our(), or local(). Simply suppress the warning if you want to use the feature, but know that in doing so you are taking the risk of using an experimental feature which may change or be removed in a future Perl version:
no warnings "experimental::declared_refs";
use feature "declared_refs";
$fooref = my \$foo;As of Perl 5.25.9, all new deprecations will come with a version in which the feature will disappear. And with a few exceptions, most existing deprecations will state when they'll disappear. As such, most deprecation messages have changed.
Improve error for missing tie() package/method. This brings the error messages in line with the ones used for normal method calls, despite not using call_method().
Make the sysread()/syswrite/() etc :utf8 handle warnings default. These warnings were under 'deprecated' previously.
'do' errors now refer to 'do' (not 'require').
Details as to the exact problem have been added to the diagnostics that occur when malformed UTF-8 is encountered when trying to convert to a code point.
Executing undef $x where $x is tied or magical no longer incorrectly blames the variable for an uninitialized-value warning encountered by the tied/magical code.
Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/%s/
The word "here" has been added to the message that was raised in v5.25.1. This is to indicate that there are contexts in which unescaped left braces are not (yet) illegal.
All changes but default_inc_excludes_do are cperl-only.
Those two new Config keys are needed for properly identifying the true compiler header and library search paths. The old libpth just assumed paths based in the calculated incpth. Several compilers have not matching search paths, which lead to wrong precedence orderings of e.g. /usr/local/ vs /opt/local, conflicting in ccflags and ldflags. syslibpth is now computed. The new keys don't add extra ccflags or ldflags directcories if they already exist in the compiler search paths. See [cperl #267] for more. Note that this doesn't fix other ordering issues, as -I and -L is evaluated before the system search paths.
This fixes serious problems with updated libraries in /usr/local/lib on macports, but probably everywhere with bad compiler search paths.
Added a post-dynamic rule for Storable, to probe for the max. stacksize and re-compile it with that.
Fixed dlopen probe and compilation with c++.
The dlopen probe with C++ needs -fPIC -shared, otherwise dlopen() will not be found. This will set ld=ld, leading to the problem below:
ld -f may not be used without -shared
Check ld=ld (caused by the failing dlopen probe from above) and ldflags without -shared and disable adding -fstack-protector to it.
Added default_inc_excludes_dot to perl -V and myconfig.
Probe for arflags, which do support D for deterministic static archives without member timestamps. On darwin we currently only have llvm-ar-mp-3.9 (since 3.7) which does support this.
ranlib is probed for the -D flag for reproducible build determinism.
With Intel long double it didn't clean random excess garbage bytes after the 10th byte.
Added the following release scripts to Porting: do-conf-clean do-conf-cperl-release do-make-cperl-release do-make-srctarball perl_version for Linux (debian and rpm) and Darwin.
Those builds are now reproducible, see below.
cperl has now support for automatic reproducible builds on most platforms. A new cf_epoch config key was added.
The config key cf_time is now based on: 1. SOURCE_DATE_EPOCH, 2. with .git the newest file in the repository, or 3. the newest file in the MANIFEST.
Builds are done with LC_ALL=C and PERL_HASH_SEED=0, but builds are still LANGUAGE or compiler specific.
Those builds are reproducible when done on the same machine and user. Otherwise set the keys: cf_by, cf_email, mydomain, myhostname, myuname also.
See [cperl #169].
This suspicious Config key was removed from cperl. If you have a NIS database use ypcat passwd. passcat is not used in any public CPAN module.
Ask for fake_signatures being compiled in as default or not. Defaults to yes with cperl, no without. Sets $Config{fake_signatures} and defines PERL_FAKE_SIGNATURE.
Probe for llabs() needed for PERL_IABS on 32bit with -Duse64bitint, the default on mingw/cygwin. Defines HAS_LLABS.
Probe for setenv() needed on some platforms with strict linkage or -fvisibility=hidden.
Probe for __attribute__((always_inline)), which is helpful with clang -flto=thin for exported mathoms (b) and inlined functions.
The problem is that __attribute__((used)) functions are not inlined. With always_inline + global visibility, but not __attribute__((used)) we get inlined variants plus exported copies for the API. Add PERL_MATHOM_CALLCONV to use it.
Added a new sanitize_address config entry and probe, and matching USE_SANITIZE_ADDRESS config.h definition.
Added a new d_attribute_used config entry and probe, and matching HASATTRIBUTE_USED config.h definition.
Added a new i_netinet_in_systm config entry and probe, and matching config.h define for Socket.
Removed the i_netinet6_in6 Config entry and probe, and matching I_NETINET6_I6 config.h define, which was a typo. This was added with cperl-5.22.2 and was never used due to the typo. It cannot be used due to RFC 2553.
Fixed the __builtin_prefetch probe, not yet used.
Added a new __builtin_ctz probe, $Config{d_builtin_ctz} key, used for faster DO_HSPLIT() calculations. About 30% faster for hash intensive tests.
Fixed t/re/regexp.t handling of compile-time errors, if combined with other result flags. cperl only.
Added t/comp/parser_run.t for parser tests requiring test.pl.
Added t/lib/warnings/toke_l1 for toke warnings tests with binary characters.
Added t/porting/embedcpp.t to check for perl.h C++ compatibility with a modern C++ compiler. There must be no fatal compilation errors in the -c step from C++ incompatibilities in any perl header file.
Note that Microsoft Visual C++ still throws errors. You cannot use that yet. See [cperl #227].
This is related to the new -Wc++11-compat and -fpermissive fixes. cperl only.
make minitest has been vastly improved. cperl only.
Fix tests for the optional -DNODEFAULT_SHAREKEYS configuration. cperl only.
Relax some timing sensitive smoker failures on overly slow systems, such as darwin on Travis with DEBUGGING. cperl only.
Time-HiRes: skip nanosleep test,
Sys-Syslog with not responding syslogd.
Removed all the old ' package seperators, and lexicalized most internal variables from c2ph. But it is not yet completely strict safe. See "Removed ' as package seperator" and [cperl #217].
Support for compiling perl on Windows using Microsoft Visual Studio 2015 (containing Visual C++ 14.0) has been added.
This version of VC++ includes a completely rewritten C run-time library, some of the changes in which mean that work done to resolve a socket close() bug in [perl #120091 and [perl #118059 is not workable in its current state with this version of VC++. Therefore, we have effectively reverted that bug fix for VS2015 onwards on the basis that being able to build with VS2015 onwards is more important than keeping the bug fix. We may revisit this in the future to attempt to fix the bug again in a way that is compatible with VS2015.
These changes do not affect compilation with GCC or with Visual Studio versions up to and including VS2013, i.e. the bug fix is retained (unchanged) for those compilers.
Note that you may experience compatibility problems if you mix a perl built with GCC or VS <= VS2013 with XS modules built with VS2015, or if you mix a perl built with VS2015 with XS modules built with GCC or VS <= VS2013. Some incompatibility may arise because of the bug fix that has been reverted for VS2015 builds of perl, but there may well be incompatibility anyway because of the rewritten CRT in VS2015 (e.g. see discussion at http://stackoverflow.com/questions/30412951).
Tweaks for Win32 VC vs GCC detection makefile code. This fixes issue that CCHOME depends on CCTYPE, which in auto detect mode is set after CCHOME, so CCHOME uses the uninit CCTYPE var. Also fix else vs .ELSE in makefile.mk
Fix some breakage, add 'undef' value for default_inc_excludes_dot in build scripts.
Preserve the Systemroot env var during env wipe for Win32 in t/op/magic.t
This fixes a test fail with VC 2005 and VC 2008 on WinXP. The Systemroot env var is required on WinXP to load SXS tracked DLLs, VC 2005 and 2008's MS libc's are SXS tracked (before and after are not), so once %ENV is wiped and systemroot is deleted the require Win32 cant load the XS DLL because the XS DLL is linked against the SXS tracked libc specific to 2005/2008.
Added strupr() and more ENV_IS_CASELESS helper functions for non-Win32/Netware builds with ENV_IS_CASELESS being defined to easier test caseless windows ENV handling on non-windows platforms.
fp definitions have been updated.
Compilation under mingw (32 and/or 64bit) was fixed by removing a duplicate pp_sys.c entry in win32/GNUMakefile.
Drop support for Linux a.out Linux has used ELF for over twenty years.
Check for null in pp_ghostent et al.
On some platforms (such as Gentoo Linux with torsocks), hent->h_aliases (where hent is a struct hostent *) may be null after a gethostent call.
The path separator for the PERL5LIB and PERLLIB environment entries is now a colon (:) when running under a Unix shell. There is no change when running under DCL (it's still |).
Remove some VMS-specific hacks from showlex.t. These were added 15 years ago, and are no longer necessary for any VMS version now supported.
Move _pDEPTH and _aDEPTH after config.h otherwise DEBUGGING may not be defined yet.
VAXC has not been a possibility for a good long while, and the versions of the DEC/Compaq/HP/VSI C compiler that report themselves as "DEC" in a listing file are 15 years or more out-of-date and can be safely desupported.
Fix some breakage, add 'undef' value for default_inc_excludes_dot in build scripts.
The hints for Hurd have been improved enabling malloc wrap and reporting the GNU libc used (previously it was an empty string when reported). [perl #128954]
VAX floating point formats are now supported.
Similar to darwin with v5.25.2c the do_open and do_close macros are now undefined on clang++, which FreeBSD uses. do_close clashes on C++ with locale. We need to use the fullname Perl_do_open and Perl_do_close functions whenever perl needs to be embedded into C++ projects. See [cperl #227]
Also affected is modperl_io.c, which is not used with C++.
t/uni/overload.t: Skip hanging test on FreeBSD.
Fixed setting MACOSX_DEPLOYMENT_TARGET=10.3 for OS X 10.3.x - 10.5.x. This is irrelevant for cperl since cperl uses ldflags and cflags -mmacosx-version-min=10.x
Don't treat -Dprefix=/usr as special, instead require an extra option -Ddarwin_distribution to produce the same results.
OS X El Capitan doesn't implement the clock_gettime() or clock_getres() APIs, emulate them as necessary.
Deprecated syscall(2) on macOS 10.12.
Several tests have been updated to work (or be skipped) on EBCDIC platforms.
Net::Ping UDP test is skipped on HP-UX.
OpenBSD 6 still does not support returning pid, gid or uid with SA_SIGINFO. Make sure this is accounted for.
With the SunPro cc compiler we use now the __global declaration for exported functions, similar to Windows __declspec(dllexport).
This is hidden under the new GLOBAL macro.
Fixed many instances of wrong or missing dVAR declarations, relevant for -Accflags=-DPERL_GLOBAL_STRUCT_PRIVATE.
The op_class() API function has been added. This is like the existing OP_CLASS() macro, but can more accurately determine what struct an op has been allocated as. For example OP_CLASS() might return OA_BASEOP_OR_UNOP indicating that ops of this type are usually allocated as an OP or UNOP; while op_class() will return OPclass_BASEOP or OPclass_UNOP as appropriate.
The output format of the op_dump() function (as used by perl -Dx) has changed: it now displays an "ASCII-art" tree structure, and shows more low-level details about each op, such as its address and class.
The initial indent level is controlled by the global PL_dumpindent, which is 4, and in Devel::Peek and B::C set to 2. With cperl the indent-level between the ops is hardcoded to 2, in perl5 to 4.
The lexer toke.c is now guaranteed to work on a linestr buffer SV with minimal PTRSIZE (4 or 8 byte) length, which enables to use fast word-wise comparison memcmp builtins. cperl only.
The negate op has now it's own ck_negate callback.
Forbid attaching magic to 4 builtin SV sentinels.
The unique sv_undef, sv_yes, sv_no and sv_placeholder SV values are compared by pointer. Adding magic to it will break that and is now silently skipped. If you need to magicalize sv_yes or sv_no, turn it into a normal IV before.
New public SV_YES and SV_NO macros were added, representing &PL_sv_yes and &PL_sv_no, plus new macros for CORE only: UNDEF and PLACEHOLDER. They cannot be used in headers, as they are used with extensions also.
Two internal debugging helpers av_dump and hv_dump had been added, to print all array elements and hash keys (and optional values).
New versions of macros like isALPHA_utf8 and toLOWER_utf8 have been added, each with the suffix _safe, like isSPACE_utf8_safe. These take an extra parameter, giving an upper limit of how far into the string it is safe to read. Using the old versions could cause attempts to read beyond the end of the input buffer if the UTF-8 is not well-formed, and ther use now raises a deprecation warning. Details are at "Character classification" in perlapi.
Calling macros like isALPHA_utf8 on malformed UTF-8 have issued a deprecation warning since Perl v5.18. They now die. Similarly, macros like toLOWER_utf8 on malformed UTF-8 now die.
Calling the functions utf8n_to_uvchr and its derivatives, while passing a string length of 0 is now asserted against in DEBUGGING builds, and otherwise returns the Unicode REPLACEMENT CHARACTER. If you have nothing to decode, you shouldn't call the decode function.
The functions utf8n_to_uvchr and its derivatives now return the Unicode REPLACEMENT CHARACTER if called with UTF-8 that has the overlong malformation, and that malformation is allowed by the input parameters. This malformation is where the UTF-8 looks valid syntactically, but there is a shorter sequence that yields the same code point. This has been forbidden since Unicode version 3.1.
The functions utf8n_to_uvchr and its derivatives now accept an input flag to allow the overflow malformation. This malformation is when the UTF-8 may be syntactically valid, but the code point it represents is not capable of being represented in the word length on the platform. What "allowed" means in this case is that the function doesn't return an error, and advances the parse pointer to beyond the UTF-8 in question, but it returns the Unicode REPLACEMENT CHARACTER as the value of the code point (since the real value is not representable).
The meanings of some internal SV flags have been changed
OPpRUNTIME, SVpbm_VALID, SVpbm_TAIL, SvTAIL_on, SvTAIL_off, SVrepl_EVAL, SvEVALED
Change hv_fetch(…, "…", …, …) to hv_fetchs(…, "…", …)
The dual-life dists all use Devel::PPPort, so they can use this function even though it was only added in 5.10.
The function "utf8n_to_uvchr" in perlapi has been changed to not abandon searching for other malformations when the first one is encountered. A call to it thus can generate multiple diagnostics, instead of just one.
A new function, "utf8n_to_uvchr_error" in perlapi, has been added for use by modules that need to know the details of UTF-8 malformations beyond pass/fail. Previously, the only ways to know why a sequence was ill-formed was to capture and parse the generated diagnostics, or to do your own analysis.
Several new functions for handling Unicode have been added to the API: "is_strict_utf8_string" in perlapi, "is_c9strict_utf8_string" in perlapi, "is_utf8_string_flags" in perlapi, "is_strict_utf8_string_loc" in perlapi, "is_strict_utf8_string_loclen" in perlapi, "is_c9strict_utf8_string_loc" in perlapi, "is_c9strict_utf8_string_loclen" in perlapi, "is_utf8_string_loc_flags" in perlapi, "is_utf8_string_loclen_flags" in perlapi, "is_utf8_fixed_width_buf_flags" in perlapi, "is_utf8_fixed_width_buf_loc_flags" in perlapi, "is_utf8_fixed_width_buf_loclen_flags" in perlapi.
These functions are all extensions of the is_utf8_string_*() functions, that apply various restrictions to the UTF-8 recognized as valid.
A new API function sv_setvpv_bufsize() allows simultaneously setting the length and allocated size of the buffer in an SV, growing the buffer if necessary.
A new API macro SvPVCLEAR() sets its SV argument to an empty string, like Perl-space $x = '', but with several optimisations.
All parts of the internals now agree that the sassign op is a BINOP; previously it was listed as a BASEOP in regen/opcodes, which meant that several parts of the internals had to be special-cased to accommodate it. This oddity's original motivation was to handle code like $x ||= 1; that is now handled in a simpler way.
Several new internal C macros have been added that take a string literal as arguments, alongside existing routines that take the equivalent value as two arguments, a character pointer and a length. The advantage of this is that the length of the string is calculated automatically, rather than having to be done manually. These routines are now used where appropriate across the entire codebase.
Note that these upstream changes are still inferior to the cperl specific memEQc and strEQc macros, which do faster word-wordwise comparisons at run-time.
The code in gv.c that determines whether a variable has a special meaning to Perl has been simplified.
The DEBUGGING-mode output for regex compilation and execution has been enhanced.
Several macros and functions have been added to the public API for dealing with Unicode and UTF-8-encoded strings. See "Unicode Support" in perlapi.
Use my_strlcat() in locale.c. While strcat() is safe in this context, some compilers were optimizing this to strcpy() causing a porting test to fail that looks for unsafe code. Rather than fighting this, we just use my_strlcat() instead.
Perl no longer panics when switching into some locales on machines with buggy strxfrm() implementations in their libc. [perl #121734]
Support for clang -flto and the new -flto=thin optimization was added, via GLOBAL declaration and __attribute__global__ for global visibility for all exported API functions, even if not used, and -DLTO. Note that is not needed for gcc -flto, and the clang variant produces slower code.
Rudimentary support for -fsanitize=cfi was also added, which is safer than the old -fstack-protector-strong, but this is not yet finished.
ExtUtils::ParseXS adds now a correct visibility declaration of XS_EXTERNAL for -flto and -fvisibility=hidden, which is needed for -fsanitize=cfi support.
Perl_xs_handshake is now properly exported, which is needed for clang -flto=thin.
XS_EXTERNAL and XSPROTO were simplified to use the new GLOBAL declaration and __attribute__global__ attribute, for easier platform abstractions.
Added many OP read-write field accessor macros, like OpFIRST, OpLAST, OpOTHER, OpKIDS, OpSPECIAL, OpSTACKED, OpDEREF, OpWANT_VOID, OpWANT_SCALAR, OpWANT_LIST. And shorter type checks: IS_TYPE, ISNT_TYPE, NO_OP_TYPE_OR_WASNT.
rpeep uses now consistently the local o, and not the global PL_op variable. See [cperl #219].
"repeatcpy" in perlapi changed the type of the 4th count argument from IV to UV.
Added a new "newPADNAMEpvn_flags" in perlapi function which disables UTF8 via flags of 0, a new "PadnameUTF8" in perlapi macro, and new PADNAMEt_UTF8 and padadd_UTF8 bits.
The maximal size of hashes has been reduced from 63 bit back to 32 bit on 64-bit systems, as with perl5 upstream and as with cperl-5.22. The only problem with 63 bit was the performance overhead of having to calculate 64-bit hashes for each string, which was not worth it. For overlarge hashes use tie to an external library which handle bigger sizes and external maps.
This affects xhv_keys, xhv_max, xhv_riter, xhv_fill_lazy, placeholders and the return values and arguments of most hv_ functions and macros. xhv_riter is now a full U32, thus the previous tombstone value -1 is now U32_MAX, so contrary to perl5 you can still iterate over the full keys range, and not just the half of it.
PL_maxo is now tracked/incremented in custom_op_register().
The static number of OPs is determined by the static MAXO definition, but users can add custom ops.
Note that perl5.25.4 removes the dynamic part maxo. We find it useful, as only maxo returns the number of current ops.
HVhek_MASK is now only 0x03, sames as HVhek_ENABLEHVKFLAGS, which is not needed anymore.
HVhek_MASK is only needed during hash collision comparisons. There we only need the 2 HEK UTF8 bits: HVhek_UTF8 and HVhek_WASUTF8, but not the 3 others: UNSHARED, TAINTED, STATIC. (the 2 last being cperl-only)
Fixed [perl #131101] crash with something like
map /x/g, (%h = ("y", 0)), (%h = ("y", 0))cperl-only
The cperl packaging recipe in do-make-cperl-release allowed empty directories to be created, which is a huge problem with debian/rpm like installations, potentially removing all files and subdirectories manually installed via cpan into sitelib and sitearch when updating or removing the package.
These list assignments now properly type check:
package Bla;
my Bla @a;
my int $i = $a[0];
# => Type of scalar assignment to $i must be int (not Bla)
$i = shift @a;
# => Type of scalar assignment to $i must be int (not Bla)
# implicit shift
(my int $j) = @a;
# => Type of list assignment to $j must be int (not Bla)i.e. we descend into many list (array/hash) ops. mderef not yet.
Also more builtin variables are now type-checked, such as @ARGV as :Array(:Str) and as :Str $ARGV, $0 and $^X. Previously only $^O. These added types do now more internal type optimizations, e.g. using s_eq instead of the generic eq when comparing with constants.
See [cperl #258].
Implemented a workaround for the debugger (-d) to step into most functions with signatures. Until the root cause with debugging tailcalls is fixed, we convert back a signature to old-style assignments under the debugger. This was the only perl5 regression. See [cperl #167].
perl5 is compileable with recent C++ compilers, but cperl since v5.24.0c not so. We fixed more wrong goto's and wrong const declarations leading to C++ errors with -fpermissive, which is basically used as stricter C mode.
E.g. error: invalid conversion from 'const HEK* {aka const hek*}' to 'HEK* {aka hek*} [-fpermissive]. error: jump to label 'float_ipow'. crosses initialization of 'unsigned int diff'
Regression since v5.24.0c, a cperl problem only. [cperl #224]
$-{$name} leak $-{$name} would leak an AV on each access if the regular expression had no named captures. The same applies to access to any hash tied with Tie::Hash::NamedCapture and all => 1. [perl #130822]
split ' ' now handles the argument being split correctly when in the scope of the unicode_strings feature. Previously, when a string using the single-byte internal representation contained characters that are whitespace by Unicode rules but not by ASCII rules, it treated those characters as part of fields rather than as field separators. This resolves [perl #130907].
Attempting to use the deprecated variable $# as the object in an indirect object method call could cause a heap use after free or buffer overflow. [perl #129274]
When checking for an indirect object method call in some rare cases the parser could reallocate the line buffer but then continue to use pointers to the old buffer. [perl #129190]
Supplying a glob as the format argument to "formline" in perlfunc would cause an assertion failure. [perl #130722]
Code like $value1 =~ qr/.../ ~~ $value2 would have the match converted into a qr// operator, leaving extra elements on the stack to confuse any surrounding expression. [perl #130705]
Since 5.24.0 in some obscure cases, a regex which included code blocks from multiple sources (e.g. via embedded via qr// objects) could end up with the wrong current pad and crash or give weird results. [perl #129881]
Occasionally local()s in a code block within a patterns weren't being undone when the pattern matching backtracked over the code block. [perl #126697]
Using substr() to modify a magic variable could access freed memory in some cases. [perl #129340]
Perl 5.25.9 was fixed so that under use utf8, the entire Perl program is checked that the UTF-8 is wellformed. It turns out that several edge cases were missed, and are now fixed. [perl #126310] was the original ticket.
No multiple evaluation of its argument anymore.
Fixed an assertion error with DEBUGGING builds in the ref op, detected with -T in pod testing. Turn off taint magic for the return value of ref with fixed ref names. [cperl #254].
Fix c3 ISA linearization with deleted ISA elements, leaving "main" plus an empty name.
cperl -Mmro=c3 -e'
@ISACLEAR::ISA=qw/XX YY ZZ/;
$ISACLEAR::ISA[1]=undef;
print join",",@{mro::get_linear_isa('ISACLEAR')}'
=> XX,main,,ZZ # wrongFix and explain sv_dump (also via Devel::Peek Dump) that maxnested means also maxelems, by printing now "... (skipping Elt 5-20)". [cperl #243].
Under use utf8, the entire Perl program is now checked that the UTF-8 is wellformed. [perl #126310].
Handle SvIMMORTALs in LHS of list assign. [perl #129991]
[perl #130010] a5540cf breaks texinfo
This involved user-defined Unicode properties.
\N{ in regcomp.An unclosed \N{ could give the wrong error message "\N{NAME} must be resolved by the lexer".
List assignment in list context where the LHS contained aggregates and where there were not enough RHS elements, used to skip scalar lvalues. Previously, (($a,$b,@c,$d) = (1)) in list context returned ($a); now it returns ($a,$b,$d). (($a,$b,$c) = (1)) is unchanged: it still returns ($a,$b,$c). This can be seen in the following:
sub inc { $_++ for @_ }
inc(($a,$b,@c,$d) = (10))Formerly, the values of ($a,$b,$d) would be left as (11,undef,undef); now they are (11,1,1).
The basic problem is that code like this: /(?{ s!!! })/ can trigger infinite recursion on the C stack (not the normal perl stack) when the last successful pattern in scope is itself. Since the C stack overflows this manifests as an untrappable error/segfault, which then kills perl.
We avoid the segfault by simply forbidding the use of the empty pattern when it would resolve to the currently executing pattern.
Avoid reading beyond the end of the line buffer when there's a short UTF-8 character at the end. [perl #128997]
do "a\0b" fail silently instead of throwing.A sub containing a "forward" declaration with the same name (e.g., sub c { sub c; }) could sometimes crash or loop infinitely. [perl #129090]
A crash in executing a regex with a floating UTF-8 substring against a target string that also used UTF-8 has been fixed.
Previously, a shebang line like #!perl -i u could be erroneously interpreted as requesting the -u option. This has been fixed.
The regex engine was previously producing incorrect results in some rare situations when backtracking past a trie that matches only one thing; this showed up as capture buffers ($1, $2, etc) erroneously containing data from regex execution paths that weren't actually executed for the final match. [perl #129897]
Certain regexes making use of the experimental regex_sets feature could trigger an assertion failure. This has been fixed.
Invalid assignments to a reference constructor (e.g., \eval=time) could sometimes crash in addition to giving a syntax error. [perl #125679]
The parser could sometimes crash if a bareword came after evalbytes. [perl #129196]
Autoloading via a method call would warn erroneously ("Use of inherited AUTOLOAD for non-method") if there was a stub present in the package into which the invocant had been blessed. The warning is no longer emitted in such circumstances. [perl #47047]
The use of splice on arrays with nonexistent elements could cause other operators to crash. [perl #129164]
Fixed case where re_untuit_start will overshoot the length of a utf8 string. [perl #129012]
Handle CXt_SUBST better in Perl_deb_stack_all, previously it wasn't checking that the current cx is the right type, and instead was always checking the base cx (effectively a noop). [perl #129029]
Fixed two possible use-after-free bugs in Perl_yylex. Perl_yylex maintains up to two pointers into the parser buffer, one of which can become stale under the right conditions.
Fixed a crash with s///l where it thought it was dealing with UTF-8 when it wasn't. [perl #129038]
Fixed place where regex was not setting the syntax error correctly. [perl #129122]
The &. operator (and the & operator, when it treats its arguments as strings) were failing to append a trailing null byte if at least one string was marked as utf8 internally. Many code paths (system calls, regexp compilation) still expect there to be a null byte in the string buffer just past the end of the logical string. An assertion failure was the result. [perl #129287]
Check pack_sockaddr_un()'s return value because pack_sockaddr_un() silently truncates the supplied path if it won't fit into the sun_path member of sockaddr_un. This may change in the future, but for now check the path in thesockaddr matches the desired path, and skip if it doesn't. [perl #128095]
Make sure PL_oldoldbufptr is preserved in scan_heredoc(). In some cases this is used in building error messages. [perl #128988]
Check for null PL_curcop in IN_LC() [perl #129106]
Fixed the parser error handling for an ':attr(foo' that does not have an ending ')'.
Fix Perl_delimcpy() to handle a backslash as last char, this actually fixed two bugs, [perl #129064] and [perl #129176].
[perl #129267] rework gv_fetchmethod_pvn_flags separator parsing to prevent possible string overrun with invalid len in gv.c
Problems with in-place array sorts: code like @a = sort { ... } @a, where the source and destination of the sort are the same plain array, are optimised to do less copying around. Two side-effects of this optimisation were that the contents of @a as visible to to sort routine were partially sorted, and under some circumstances accessing @a during the sort could crash the interpreter. Both these issues have been fixed, and Sort functions see the original value of @a.
pack("p", ...) used to emit its warning ("Attempt to pack pointer to temporary value") erroneously in some cases, but has been fixed.
@DB::args is now exempt from "used once" warnings. The warnings only occurred under -w, because warnings.pm itself uses @DB::args multiple times.
The use of built-in arrays or hash slices in a double-quoted string no longer issues a warning ("Possible unintended interpolation...") if the variable has not been mentioned before. This affected code like qq|@DB::args| and qq|@SIG{'CHLD', 'HUP'}|. (The special variables @- and @+ were already exempt from the warning.)
gethostent and similar functions now perform a null check internally, to avoid crashing with torsocks. This was a regression from 5.22. [perl #128740]
defined *{'!'}, defined *{'['}, and defined *{'-'} no longer leak memory if the typeglob in question has never been accessed before.
In 5.25.4 fchown() was changed not to accept negative one as an argument because in some platforms that is an error. However, in some other platforms that is an acceptable argument. This change has been reverted [perl #128967].
Mentioning the same constant twice in a row (which is a syntax error) no longer fails an assertion under debugging builds. This was a regression from 5.20. [perl #126482]
Many issues relating to printf "%a" of hexadecimal floating point were fixed. In addition, the "subnormals" (formerly known as "denormals") floating point anumbers are now supported both with the plain IEEE 754 floating point numbers (64-bit or 128-bit) and the x86 80-bit "extended precision". Note that subnormal hexadecimal floating point literals will give a warning about "exponent underflow". [perl #128843], [perl #128889], [perl #128890], [perl #128893], [perl #128909], [perl #128919].
A regression in 5.24 with tr/\N{U+...}/foo/ when the code point was between 128 and 255 has been fixed. [perl #128734].
A regression from the previous development release, 5.23.3, where compiling a regular expression could crash the interpreter has been fixed. [perl #128686].
Use of a string delimiter whose code point is above 2**31 now works correctly on platforms that allow this. Previously, certain characters, due to truncation, would be confused with other delimiter characters with special meaning (such as ? in m?...?), resulting in inconsistent behaviour. Note that this is non-portable, and is based on Perl's extension to UTF-8, and is probably not displayable nor enterable by any editor. [perl #128738]
@{x followed by a newline where x represents a control or non-ASCII character no longer produces a garbled syntax error message or a crash. [perl #128951]
An assertion failure with %: = 0 has been fixed. [perl #128238]
In Perl 5.18, the parsing of "$foo::$bar" was accidentally changed, such that it would be treated as $foo."::".$bar. The previous behavior, which was to parse it as $foo:: . $bar, has been restored. [perl #128478]
Since Perl 5.20, line numbers have been off by one when perl is invoked with the -x switch. This has been fixed. [perl #128508]
Vivifying a subroutine stub in a deleted stash (e.g., delete $My::{"Foo::"}; \&My::Foo::foo) no longer crashes. It had begun crashing in Perl 5.18. [perl #128532]
Some obscure cases of subroutines and file handles being freed at the same time could result in crashes, but have been fixed. The crash was introduced in Perl 5.22. The cperl fix doesn't hurt run-time performance as the perl5.26 fix does. [perl #128597]
Code that looks for a variable name associated with an uninitialized value could cause an assertion in cases where magic is involved, such as $ISA[0][0]. This has now been fixed. [perl #128253]
A crash caused by code generating the warning "Subroutine STASH::NAME redefined" in cases such as sub P::f{} undef *P::; *P::f =sub{}; has been fixed. In these cases, where the STASH is missing, the warning will now appear as "Subroutine NAME redefined". [perl #128257]
Fixed an assertion triggered by some code that handles deprecated behavior in formats, e.g. in cases like this:
format STDOUT =
@
0"$x"A possible divide by zero in string transformation code on Windows has been avoided, fixing a crash when collating an empty string. [perl #128618]
Some regular expression parsing glitches could lead to assertion failures with regular expressions such as /(?<=/ and /(?<!/. This has now been fixed. [perl #128170]
Scalar keys %hash can now be assigned to consistently in all scalar lvalue contexts. Previously it worked for some contexts but not others.
${\vec %h, 0, 1} and ${\substr %h, 0} do not segfault anymore, rather the lvalue context is propagated, and list context properly handled. [perl #128260]
When the right side of the range is a UTF-8 encoded string, but the left side not, downgrade the right side to native octets. E.g.
my $r = chr 255; utf8::upgrade $r; my $num = ("a" .. $r);
print $numshould print 26 but does 702, because the utf-8 repr. of \x{ff} is "\303\277" [UTF8 "\x{ff}"], and the range was incremented from "a" to "\x{c3}\x{bf}" instead. See [cperl #218].
Fixed several issues with -Duseshrplib, a shared libcperl.$so: install it (!!), fix ExtUtils::Embed and B-C compilation and testing, fix tests on darwin, fix configuration probe of Term::ReadKey.
Fixed sv_dump of fbm-magic strings which did previously contain the wrong "VALID,EVALED" string for a flag which is either VALID or EVALED. cperl only.
Fixed a cperl-only failure in signatures with default blocks introducing a new lexical variable. As in sub t151($a,$b=do{my $f},$c=1){} t151($x,$x,$x). This failure was only fatal on 32bit + -Duse64bitint systems.
SIGNATURE_arg_default_op does not have a items arg. See [cperl #164]. and [cperl #213].
In scan_vstring(). See [cperl #211].
against huge data (2GB) overflows on 64bit.
We are now in a 64bit world and need to get rid of all the wrong 32bit (2GB) size limits. Some of these fixes seem to be even security relevant, as in the last 2GB series from [cperl #123].
chop/chomp of only half of overlarge arrays.
Or ~"a"x2G complement of overlarge strings, silently processing only the half - as with overlong hash keys.
There was also a smartmatch Array - CodeRef rule, which passed only over half the array elements. The Hash part was also wrong, but the wrong number was not used.
regex match group of >2GB string len.
Allow repeat count >2GB and don't silently cap it at IV_MAX. Which was at least better then silent wrap around.
Missing optimization of inplace substitution via clen overflow.
use-after-free in Dynaloader (ReadKey probe with -DDEBUG_LEAKING_SCALAR), heap-overflow in gv_fetchfile (t/comp/parser.t), heap-overflow with signatures, heap-overflow in XSLoader, invalid memEQc in toke.c missing out on parsing #!perl -opts, B-C global-buffer-overflow with dynamic COW strings, wrong savepvn args.
There are still heap-use-after-free problems with perlcc and PERL_DESTRUCT_LEVEL=2.
See [cperl #207]
Fixed overwriting the HVhek_UNSHARED bit in the hash loop broken with v5.9.
This fixed -DNODEFAULT_SHAREKEYS. In the default configuration without NODEFAULT_SHAREKEYS since 5.003_001 all hash keys are stored twice, once in the hash and once again in PL_strtab, the global string table, with the benefit of faster hash loops and copies. Almost all hashtables get the SHAREKEYS bit. With -Accflags=-DNODEFAULT_SHAREKEYS simple scripts are 20-30% faster. [cperl #201]
A HEf_SVKEY hek has no tainted flag, the pointed to SV has. This is a cperl-only security feature.
When miniperl calls csh to implement glob(), we cleared %ENV temporarily to avoid csh dying on invalid values for things like LS_COLORS. That has proven to have far too many problems, since many system-dependent env vars are necessary for calling an external process. See the [perl #126041] ticket for details.
A better solution is temporarily to clear only those vars that are known to be problematic and make csh possibly fail. There only hap- pens to be one of those at present, namely LS_COLORS.
A SEGV in mess_sv during global destruction with a DEBUGGING perl and -DS been fixed, occuring when we wanted to report the location of an error when curcop has already been freed.
Testcase: ./miniperl -DS -e '$_="f"; s/./"&".$&/ee'
A SEGV in ck_shift with an empty/wrong current function, caused by a syntax error has been fixed. The syntax error is now reported lateron. Testcase: 'qq{@{sub{q}}]]}}; s0{shift'
E.g. perl -CS -e 'use utf8; q«' prints now Can't find string terminator "«" anywhere before EOF at -e line 1.
until ($x = 1) { ... } and ... until $x = 1 now properly warn when syntax warnings are enabled. [perl #127333]
require followed by a single colon (as in foo() ? require : ... is now parsed correctly as require with implicit $_, rather than require "". [perl #128307]
For open cperl problems see [issues].
Some of these fixes also can to be backported from perl5.25.x upstream.
The perl debugger doesn't yet work fully with signatures. See e.g. [cperl #167]
t/op/taint.t contained a test with signatures and 6 default arguments, which on some 32 bit systems led to random "Reference parameter cannot take default value at op/taint.t line 2461" compile-time failures. This test has been rewritten to ony use 4 arguments.
See [cperl #164]
clang -flto=thin and on some systems even gcc -flto with -O3 or -finline leads to invisible symbols which were inlined and not exported, even if they should be declared as public API. Work is ongoing in the feature/gh186-lto-thin branch, but there the inlining is disabled by the used attribute, leading to a 10% performance regression. On the other hand a working clang-3.9 -flto leads to 20% performance improvements.
until ($x = 1) { ... } and ... until $x = 1 should warn when syntax warnings are enabled. [perl #127333]
${\vec %h, 0, 1} and ${\substr %h, 0} should not segfault, rather the lvalue context should be propagated, and list context properly handled. [perl #128260]
Parsing bad POSIX charclasses no longer leaks memory. This was fixed in Perl 5.25.2 [perl #128313]
Fixed issues with recursive regexes. The behavior was fixed in Perl 5.24.0. [perl #126182]
Jon Portnoy (AVENJ), a prolific Perl author and admired Gentoo community member, has passed away on August 10, 2016. He will be remembered and missed by all those with which he came in contact and enriched with his intellect, wit, and spirit.
cperl 5.26.0 represents approximately 1 year of development since Perl 5.24.2c and contains approximately 420,000 lines of changes across 2,800 files from 75 authors.
Excluding auto-generated files, documentation and release tools, there were approximately 260,000 lines of changes to 1,900 .pm, .t, .c and .h files.
The following people are known to have contributed the improvements that became cperl 5.26.0:
Reini Urban, Karl Williamson, David Mitchell, Father Chrysostomos, Yves Orton, Jarkko Hietaniemi, Aaron Crane, Tony Cook, Abigail, Dan Collins, Lukas Mai, Craig A. Berry, James E Keenan, Hugo van der Sanden, Andy Lester, Dagfinn Ilmari Mannsåker, Jim Cromie, Matthew Horsfall, Sawyer X, Aristotle Pagaltzis, H.Merijn Brand, Daniel Dragan, Steve Hay, Niko Tyni, Pali, Zefram, Dominic Hargreaves, Chris 'BinGOs' Williams, Renee Baecker, Ricardo Signes, E. Choroba, Petr Písař, Steffen Müller, John Lightsey, Nicolas Rochelemagne, Karen Etheridge, Shlomi Fish, Dave Rolsky, Smylers, Christian Hansen, Yaroslav Kuzmin, Misty De Meo, Ævar Arnfjörð Bjarmason, Alex Vandiver, Stefan Seifert, Tomasz Konojacki, James Raspass, Ed Avis, Steven Humphrey, Neil Bowers, Rafael Garcia-Suarez, Theo Buehler, Thomas Sibley, Doug Bell, Jerry D. Hedden, Samuel Thibault, Unicode Consortium, J. Nick Koston, Colin Newell, Sergey Aleynikov, Leon Timmermans, Maxwell Carey, Christian Millour, Chase Whitener, Pino Toscano, Peter Avalos, Salvador Fandiño, Dave Cross, Andrew Fresh, Andreas Voegele, Hauke D, Rick Delaney, François Perrad, Richard Levitte, vendethiel.
The list above is almost certainly incomplete as it is automatically generated from version control history. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker.
Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.
For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.
Generated with:
cperl Porting/acknowledgements.pl cperl-5.24.2..HEADIf you find what you think is a bug, you might check the articles recently posted to the comp.lang.perl.misc newsgroup and the perl bug database at https://rt.perl.org/ . There may also be information at http://www.perl.org/ , the Perl Home Page.
If you believe you have an unreported bug, please run the perlbug program included with your release. Be sure to trim your bug down to a tiny but sufficient test case. Your bug report, along with the output of perl -V, will be sent off to perlbug@perl.org to be analysed by the Perl porting team.
If you think it's a cperl specific bug or trust the cperl developers more please file an issue at https://github.com/perl11/cperl/issues.
If the bug you are reporting has security implications which make it inappropriate to send to a publicly archived mailing list, then see "SECURITY VULNERABILITY CONTACT INFORMATION" in perlsec For details of how to report the issue.
The Changes file for an explanation of how to view exhaustive details on what changed.
The INSTALL file for how to build Perl.
The README file for general stuff.
The Artistic and Copying files for copyright information.