perlcdelta - what is new for cperl v5.30.0
This document describes differences between the cperl 5.28.2 and the cperl 5.30.0 release.
If you are upgrading from an earlier release such as v5.26.2c, first read perl5280cdelta, which describes the differences between v5.28.0c and v5.26.2c.
The second argument to bless, the new classname, is now checked to be a valid package name identifier when strict 'names' is in effect, and the name is now normalized. [cperl #376]
This is a bugfix, but a major compatibility change.
When a subroutine with the same name of a package exists, all package methods are rerouted through this subroutine, effectively eliminating all access to the package via methods. This is almost always an error, unless you want to override access to the package or want a data-only package, without any methods. cperl added the shadow warnings "Subroutine &%s::%s masks existing package %s" in perldiag, "Subroutine &%s masks new package %s" in perldiag and "Subroutine &%s masks new class %s" in perldiag. Use no warnings 'shadow'; to silence these new warnings.
A new ffi was added to core, with a syntax resembling perl6 NativeCall. Perl6 traits are cperl attributes, but attribute values may be deferred to run-time. Packagers need to add libffi to their library dependencies. Windows packages are provided, see appveyor.yml.
Not yet implemented are the :encoded() attribute for string arguments and return values, the various encoded string types Uni, wchar, c-struct helpers, native arrays and native classes (i.e. structs), callback helpers and support for variadic (varargs) functions.
Run-time type violations a throw a new type warning, which can be made fatal with use types 'strict' or use warnings 'FATAL' => 'types'; "Type of arg %s to %s should be %s (not %s)" in perldiag. The compile-time error uses the word "must" instead of "should". This might be a problem when typing an argument to str or Str, but the argument is stringified via overload from a class, e.g. File::Temp::Dir. This case, when the argument is not of str|Str, but a user-defined class with a stringify overload is not yet handled.
[cperl #389] and [cperl #387]. cperl-only
When roles are copied from a class to another, we can now handle XS methods and methods without internal lexical variables better, by simple GV aliasing.
The other methods are now properly cloned and the $self type is adjusted to the new class for the created new method, because v5.30 started argument type checking at run-time.
cperl can also now adjust field indices when composing roles, when the indices of the used lexical variables of the two methods do not match, by using the new internal experimental inliner. [cperl #311] The error "panic: cannot yet adjust field indices when composing role %s::%s into %s %s [cperl #311]" is gone in the general case.
You can now do something like this in a regular expression pattern
qr! \p{nv= /(?x) \A [0-5] \z / }!which matches all Unicode code points which have numeric value is between 0 and 5 inclusive.
This marks another step in implementing the regular expression features the Unicode Consortium suggests.
Most properties are supported, with the remainder planned for 5.32. Details are in "Wildcards in Property Values" in perlunicode.
Previously it was an error to evaluate a named character \N{...} within a single quoted regular expression pattern (whose evaluation is deferred from the normal place). This restriction is now removed.
For details, see https://www.unicode.org/versions/Unicode12.1.0/.
The Word_Break property, as in past Perl releases, remains tailored to behave more in line with expectations of Perl users. This means that sequential runs of horizontal white space characters are not broken apart, but kept as a single run. Unicode 11 changed from past versions to be more in line with Perl, but it left several white space characters as causing breaks: TAB, NO BREAK SPACE, and FIGURE SPACE (U+2007). We have decided to continue to use the previous Perl tailoring with regards to these.
As of July 2018, another aspect of Unicode 11.0 is causing problems, and that is the changing the Georgian script to have both lower and uppercase letters. It turns out that many devices do not have fonts installed that can display the new uppercase letters, and people using Georgian are finding their text more or less illegible. It is being suggested that implementations delay their adoption of this portion of 11.0. But since the next stable release of Perl is almost a year away, this aspect of 11.0 is being left in, with the option to back it out if things haven't improved by that time, or if we get too many complaints about the development release being unusable for Georgian users.
Because of a change in Unicode release cycles, Perl jumps from Unicode 10.0 in Perl 5.28 to Unicode 12.0 in Perl 5.30.
There are few new scripts: Elymaic, Nandinagari, Nyiakeng_Puachue_Hmong (Hmnp), Wancho, all of them not available for identifiers.
Allow a rational number specified in looking up the Numeric Value Unicode property to not be in lowest possible terms.
The Numeric Value property allows one to find all code points that have a certain numeric value. An example would be to match against any character in any of the world's scripts which is effectively equivalent to the digit zero.
We accept either integers (like \p{nv=9}) or rationals (like \p{nv=1/2}). But we also accept floating point representations in case a conversion to numeric has happened, (like \p{nv=0.5}). The necessary precision is now automatically pre-calculated and the lookup normalized.
"n" specifiable in a regular expression quantifier of the form "{m,n}" has been doubled to 65534The upper limit "n" specifiable in a regular expression quantifier of the form "{m,n}" has also been doubled to 65534.
The meaning of an unbounded upper quantifier "{m,}" remains unchanged. It matches 2**31 - 1 times on most platforms, and more on ones where a C language short variable is more than 4 bytes long.
-Drv now means something on -DDEBUGGING buildsNow, adding the verbose flag (-Dv) to the -Dr flag turns on all possible regular expression debugging, as with use re 'debug';. See re.
Eliminated recursion from internal finalize_op() and optimize_op(), avoiding segfaults with extremely nested ops. E.g. with
perl -e 'my $line = "\$cond ? \$a : \n";
my $code = ($line x 100000) . "\$b;\n";
eval $code;'Contrary to what is said in the relevant perldelta, your code is now slower. Before it crashed with a segfault, now it does much more. The regular case is also slower due to stack handling on the heap, similar to the regexp changes in 5.10. [perl #108276].
Turkic languages have different casing rules than other languages for the characters "i" and "I". The uppercase of "i" is LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130); and the lowercase of "I" is LATIN SMALL LETTER DOTLESS I (U+0131). Unicode furnishes alternate casing rules for use with Turkic languages. Previously, Perl ignored these, but now, it uses them when it detects that it is operating under a Turkic UTF-8 locale.
Previously, these calls were only used when the perl was compiled to be multi-threaded. To always enable them, add
-Accflags='-DUSE_THREAD_SAFE_LOCALE'to your Configure flags.
Detect if we are running under valgrind and set destruct_level to 2. If so, destruct all ops, leading to no valgrind leaks, even with --leak-check=full. cperl-only [cperl #400]
Using a lookbehind assertion (like (?<=foo) or (?<!bar) previously would generate an error and refuse to compile. Now it compiles (if the maximum lookbehind is at most 255 characters), but raises a warning in the new experimental::vlb warnings category. This is to caution you that the precise behavior is subject to change based on feedback from use in the field.
See "(?<=pattern)" in perlre and "(?<!pattern)" in perlre.
The list of valid scripts for identifiers was purged from all Limited Use scripts from http://www.unicode.org/reports/tr31/#Table_Limited_Use_Scripts. They fail now.
Unicode 10.0 started disallowing all Aspirational Use Scripts, but we didn't handle them at all. Table 4. Candidate Characters for Exclusion from Identifiers scripts are still allowed, and still need to be explicitly enabled as a use utf8 argument. cperl-only.
Integer arithmetic in Perl_my_setenv() could wrap when the combined length of the environment variable name and value exceeded around 0x7fffffff. This could lead to writing beyond the end of an allocated buffer with attacker supplied data.
perl5 fixed Perl_my_setenv() by allowing %ENV sizes larger than ARG_MAX (typically 128Kb), up to 2GB. This is considered a security risk. It writes onto the kernel stack, even if often caught by MAXEXECARGS "Argument list too long" kernel errors on the next exec call. cperl disallows %ENV larger than the sysconf(_SC_ARG_MAX)/MAX_ARG_STRLEN/ ARG_MAX/NCARGS size with a new "Environment size %d larger than the allowed %d" in perldiag error.
Iterating over a hash may now by default not change any key. Allow the previous destructive behavior of changing a hash while iterating over it with use hashiter; See [cperl #117], cperl-only.
$[ is fatalSetting $[ to a non-zero value has been deprecated since Perl 5.12 and now throws a fatal error. See "Assigning non-zero to $[ is fatal" in perldeprecation.
Calling sysread(), syswrite(), send() or recv() on a :utf8 handle, whether applied explicitly or implicitly, is now fatal. This was deprecated in perl 5.24.
There were two problems with calling these functions on :utf8 handles:
All four functions only paid attention to the :utf8 flag. Other layers were completely ignored, so a handle with :encoding(UTF-16LE) layer would be treated as UTF-8. Other layers, such as compression are completely ignored with or without the :utf8 flag.
sysread() and recv() would read from the handle, skipping any validation by the layers, and do no validation of their own. This could lead to invalidly encoded perl scalars.
Declarations such as my $x if 0 are no longer permitted. This feature was deprecated in 5.10 (and replaced by two better constructs, one of it backwards compatible) and is now a fatal error. [perl #133543]
Note: cperl does not agree that this feature is a bug, but this ship sailed already over a decade ago.
These special variables, long deprecated, now throw an error when used. See "$* is no longer supported as of Perl 5.30" in perldiag and "$# is no longer supported as of Perl 5.30" in perldiag. [perl #133583]
The dump() function, long discouraged, may no longer be used unless it is fully qualified, i.e., CORE::dump().
The File::Glob::glob() function, long deprecated, has been removed and now throws an exception which advises use of File::Glob::bsd_glob() instead.
pack() no longer can return malformed UTF-8It croaks if it would otherwise return a UTF-8 string that contains malformed UTF-8. This protects agains potential security threats. This is considered a bug fix as well.
There are several sets of digits in the Common script. [0-9] is the most familiar. But there are also [\x{FF10}-\x{FF19}] (FULLWIDTH DIGIT ZERO - FULLWIDTH DIGIT NINE), and several sets for use in mathematical notation, such as the MATHEMATICAL DOUBLE-STRUCK DIGITs. Any of these sets should be able to appear in script runs of, say, Greek. But the design of 5.30 overlooked all but the ASCII digits [0-9], so the design was flawed. This has been fixed, so is both a bug fix and an incompatibility. [perl #133547] [perl #133547]
All digits in a run still have to come from the same set of ten digits.
In cperl only the following special unicode groups within regexes are reserved:
Name: \N{
Property: \p{ \P{
Break: \b{ \B{
Code: \x{ \o{
Group: \g{All other /{}/ sequences are allowed in cperl and not deprecated anymore. There's no need to quote the literal \{ and \}, only if it's ambiguous and can be mixed up with those reserved unicode groups.
automake and more widespread utils does not need to be plagued by spurious "Unescaped left brace in regex" warnings, when they dont need to be quoted. We undeprecated those new warnings and errors, there's nothing more to reserve (yet). [cperl #362]
qr/\p{_Case_Ignorable}This property is no longer used in the core, nor in cpan, and is marked as for core use only, not necessarily stable. I have kept it around because it was work to remove it, but now the revamping of the property lookup scheme was causing failures with it, when compiling on early Unicode releases. That could be fixed with extra work, but simply removing it also fixes the problem and avoids future maintenance costs.
qr/\p{_Comb_Above}This property is no longer used in the core, nor in cpan, and is marked as for core use only, not necessarily stable. I have kept it around because it was work to remove it, but now the revamping of the property lookup scheme was causing failures with a similar property, and the previous commit removed that one.
qr/\p{_CanonDCIJ}The property is not used in cpan, and is being removed as part of the cleanup instigated because another of the 3 would require extra code to handle if we were to keep it around.
Optimization of IV to UV conversions. [perl #133677].
Speed up of the integer stringification algorithm by processing two digits at a time instead of one. [perl #133691].
Minor improvements based on LGTM analysis and recommendation. (https://lgtm.com/projects/g/Perl/perl5/alerts/?mode=tree). [perl #133686]. [perl #133699].
Improved Unicode numeric property lookups, for \p{nv=float}, in size and speed by using more pre-compiled perfect hashes instead of slow pure-perl invlist walks at run-time, esp. for all numeric properties (70 more). Also avoids more swash creations.
Translating from UTF-8 into the code point it represents now is done via a deterministic finite automaton, speeding it up. As a typical example, ord("\x7fff") now requires 12% fewer instructions than before. The performance of checking that a sequence of bytes is valid UTF-8 is similarly improved, again by using a dfa.
Code optimizations in regcomp.c, regcomp.h, regexec.c.
The second regcomp pass was removed.
The handling of user-defined \p{} properties (see "User-Defined Character Properties" in perlunicode) has been rewritten to be in C (instead of Perl). This removed all uses of swashes from regular expression compilation and execution.
This speeds things up, but in the process several inconsistencies and bug fixes are made.
A few error messages have minor wording changes. This is essentially because the new way is integrated into the regex error handling mechanism that marks the position in the input at which the error occurred. That was not possible previously. The messages now also contain additional back-trace-like information in case the error occurs deep in nested calls.
A user-defined property is implemented as a perl subroutine with certain highly constrained naming conventions. It was documented previously that the sub would be in the current package if the package was unspecified. This turned out not to be true in all cases, but now it is.
All recursive calls are treated as infinite recursion. Previously they would cause the interpreter to panic. Now, they cause the regex pattern to fail to compile.
Similarly, any other error likely would lead to a panic; now to just the pattern failing to compile.
The old mechanism did not detect illegal ranges in the definition of the property. Now, the range max must not be smaller than the range min. Otherwise, the pattern fails to compile.
The intention was to have each sub called only once during the lifetime of the program, so that a property's definition is immutable. This was relaxed so that it could be called once for all /i compilations, and potentially a second time for non-/i (the sub is passed a parameter indicating which). However, in practice there were instances when this was broken, and multiple calls were possible. Those have been fixed. Now (besides the /i,non-/i cases) the only way a sub can be called multiple times is if some component of it has not been defined yet. For example, suppose we have sub IsA() whose definition is known at compile time, and it in turn calls isB() whose definition is not yet known. isA() will be called each time a pattern it appears in is compiled. If isA() also calls isC() and that definition is known, isC() will be called just once.
There were some races and very long hangs should one thread be compiling the same property as another simultaneously. These have now been fixed.
Optimized the static link order of objects for the default case, when you cannot use lto, pgo, autofdo nor bolt. Added lto to the default ccflags. See http://perl11.github.io/blog/bolt.html and [cperl #381].
Improve newSVpvn_share
Don't downgrade twice, the is_utf8 flags can be taken from the returned hek, which does the downgrading to bytes already.
The oelemfast signature changed from (:Ref,:Int):Any to ():Any. The object stays now as RV (as pad) and is not taken from the stack, the index neither.
The list of new and updated modules is modified automatically as part of preparing a Perl release, so the only reason to manually add entries here is if you're summarising the important changes in the module update. (Also, if the manually-added details don't match the automatically-generated ones, the release manager will have to investigate the situation carefully.)
ffi helpers and ffi types.
pragma to allow hash iterators changing keys for back-compat. See "Protected hash iterators".
Yet unused pragma to disable the internal function inlining optimizer, via no inline;
Our new YAML::Safe has been added to the Perl core, replacing our patched version of YAML::XS.
Fix absolute path handling on VMS
Add ffi support, :native(), :encoded(), :nativeconv(), :symbol().
Fixed for -DPERL_GLOBAL_STRUCT. Add ffi support: CvXFFI, CvFFLIB. CVf_EXTERN flag
New major release, with dVAR support and experimental cross support and -m (module) support. Add $dynamic_copwarn needed for v5.29.7+ (not cperl). The new script buildcc for module support is not yet functional, only a placeholder.
Removed arybase support.
Better way to catch warnings. [cpan #126900].
Typo in pod
Added transparent lzip and zstd delegation in tests. More meta.
Add the FREEBSD_KERNEL_VERSION key for FreeBSD.
Whitespace only
Added USE_THREAD_SAFE_LOCALE
Update to 2.27-TRIAL upstream with our cperl fixes. Some of our fixes were accepted upstream.
Fixed critical rmtree bug with a symlink crossing mountpoints previous directory ~/.cpan/build changed before entering tmp-32186, expected dev=47 ino=244719549, actual dev=47 ino=33303, aborting at CPAN/Distribution.pm line 591
Add many upstream tests and distroprefs. Still need to install them. Added the cperl distroprefs.
Support YAML::Safe for distroprefs.
Add YAML::Safe support.
Make encoder independent on Math::BigInt version
Rethrow error from eval_sv and eval_pv(), e.g. when Math::BigInt/BigFloat fails.
Fix encoding Inf and NaN from PV and NV slots to JSON_TYPE_INT
Fix memory corruption in sv_to_ivuv() function
Add new method ->require_types
Fix typed json encoder conversion from scalar's PV and NV slot to JSON_TYPE_INT
Fix inconsistency with warnings in typed json encoder
Fix Perl 5.8.0 support
Fixed minor pod typo
Document invalid recursive callbacks or overloads
Fix unicode strings with BOM corrupt ->utf8 state The BOM encoding effects only its very own decode call, not its object.
Fix incr_text refcounts and add incr_rest testcase
Fix encode_stringify string-overload refcnt problem "Attempt to free unreferenced scalar" with convert_blessed and overload.
Fixed long pathnames test for getcwd() on older systems, like debian-7.11 i386, which still returns ENAMETOOLONG on large enough buffers.
Fix the HAS_GET_CURRENT_DIR_NAME check, and the is_ENAMETOOLONG macro. Note that get_current_dir_name() is now in effect, which returns the current PWD, i.e. fastcwd, and not the realpath() (i.e the abs_path) of it. This e.g. broke some DBD::File tests, when behind symlinks.
For getcwd with get_current_dir_name() - i.e. fastcwd - add abs_path. See also http://perl11.github.io/blog/fastcwd.html
Don't translate ... to ..\.. on Win32. (an old command.com limitation) [perl #123724]
Add and use Internals::getcwd().
Restore deparsing support in the XS dumper, on Perl 5.18 and earlier.
Fix bug when dumping globs with quoting (which now happens for all Unicode glob names)
Behavior change: $dumper-Useqq(undef)> is now treated as setting the "useqq" option, not getting it (and similarly for other options) [perl #113090]
Quote glob names better; notably, Unicode globs are now handled correctly. [perl #119831]
ParseOpenInfo() leaks if it croaks https://github.com/pmqs/DB_File/issues/1
Added hints files, sourced from http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/lang/perl5/files/
Expanded the documentation
Adjust Devel::Peek pod to cperl FLAGS
PL_in_sub only with v5.29.0c
Merge 3.52 with cperl improvements:
cperl croak_no_modify adds now the SV to the message.
support make --silent.
support devel VERSION with _
fix deps
sync embed.fnc with cperl core, with #ifdef USE_CPERL
sync parts with 5.30 updatesRevert "Use latest blead embed.fnc"
It changed the meaning of several flags (n=>T, s=>S,M=>x,...) just for
some apidocs troubles which are easily solved otherwise.
It breaks backcompat, and breaks the equality of core embed.fnc.
perl5.30 also rejected itmake regen is now in two modes: cperl or perl5 (as in cperl-core embed_lib)
Add croak_no_modify_sv
Remove implemented TODOs
Fix for older EUMM <6.46 which has no META_MERGE
Fix 5.6.0 tests with early Test::Harness: add done_testing, support plan => no_plan, fallback => NN for 5.6.0 (Test::Harness < 2), avoid Test.pm, skip git diff, as make regen is different there, skip 3 failing 5.6.0 tests in ppphtest
CVE-2019-11358 + CVE-2015-9251 jquery-1.12.4.js update to latest v3.4.1. Also update the two plugins to latest, and adjusted the tablesorter css.
Adjust the short %PL_DBsub keys (without main::) compatible to perl5.
Add all the tests, cleanup after the tests.
Extend dl_find_symbol on dl_win32 to mimic the POSIX behaviour with an empty libhandle argument: Search the symbol in all loaded shared libraries. This is a convenience platform-compat feature for the ffi.
Add Encode::ONLY_PRAGMA_WARNINGS by default to :encoding layer variable $PerlIO::encoding::fallback.
ONLY_PRAGMA_WARNINGS reports only those warnings which are currently enabled by pragma warnings. When ONLY_PRAGMA_WARNINGS is not set then Encode would report all warnings. ONLY_PRAGMA_WARNINGS would have no effect when flag ENCODE_WARN_ON_ERR is not set.
Make Errno_pm.PL compatible with /usr/include/<ARCH>/errno.h. Protect from including some avx512 intrinsics directly on mingw.
Added optimize to ccopts, required for -flto. Fixed documentation of ccopts. Fixed test to use ccopts, include local -I.., indentation, remove wrong -O hack.
Minor internal changes only
Added -flto support for static libs (need the lto plugin) and D for deterministic builds, using arflags for AR_STATIC_ARGS, not just 'cr'.
fix FULLPERL for win32 cperl
(perl #133654) don't include OUTLIST parameters in the prototype: The generated prototype (with PROTOTYPES: ENABLE) would include OUTLIST parameters, but these aren't arguments to the perl function.
array_base was removed.
Add note to close or flush filehandles before calling copy or move.
$File::Find::dont_use_nlink now defaults to 1 on all platforms. Fixes [perl #133673], [perl #128894], and [perl #126144].
Fatalized File::Glob::glob(), which was deprecated since 5.8. However, no deprecation message was issued; only perl5.008delta.pod and a comment in the file mention its deprecation. The warnings was added with v5.25.10.
Fix symlink abort crossing mountpoint with CPAN::Distribution previous directory ~/.cpan/build changed before entering tmp-32186, expected dev=47 ino=244719549, actual dev=47 ino=33303, aborting at CPAN/Distribution.pm line 591
Fix method of generating names for dummy users and groups during testing (i.e. under docker) [cpan #121967].
Add SECURITY AND PORTABILITY warning paragraph to pod.
Skip EINVAL on FreeBSD tests when binding a tcp listener on localhost.
Allow ETIMEDOUT on FreeBSD INET6 socket with V6Only true. [cpan #128986]
Fixed parallel testing [cpan #105700]
Probe for more external gzips, make t/external.t strict safe.
Upstream + keep some of our cperl-specific docs and test improvements.
Hint about $dir/$version is checked by lib.pm and deleted from @INC
Add to_base() and from_base() to Math::BigInt and corresponding library methods _to_base() and _from_base() to Math::BigInt::Lib. This was inspired by CPAN RT #122681.
In the documentation for each of the to_(bin|hex|oct|bytes) methods, add a reference to the corresponding from_(bin|hex|oct|bytes) method.
bnok() for Math::BigInt and Math::BigFloat now support the full Kronenburg extension. The behaviour is identical to the behaviour of the Maple and Mathematica function for negative integers n, k. Add tests. RT #95628.
Fix POD errors. RT #125141.
Move bitwise operators signed and, signed or, and signed xor from lib/Math/BigInt/CalcEmu.pm into lib/Math/BigInt/Lib.pm. The file lib/Math/BigInt/CalcEmu.pm is no longer needed and thus removed.
No code nor test changes.
Added SEE ALSO section to documentation. RT#100575 Unreachable code cleanup
Fix -Wmaybe-uninitialized warnings
Add SECURITY AND PORTABILITY warning paragraph to pod.
Features
Allow data_size > 1024, up to 65535, i.e. fragmented packets. It is recommended to stay below 1472 though for the typical 1500 MTU. Many simple devices do not allow fragmented ICMP packets (> 1472). [cpan #17409]
Bugfixes
Fix the max_datasize documentation
Test fixes
The 2 sudo tests on PERL_CORE with a shared perl lib. Skip failing freebsd localhost resolver tests.
Add SECURITY AND PORTABILITY warning paragraph to pod.
Added the new setstate and keepstate ops for the new inliner, variants of nextstate.
Restrict the new op 'enterffi'.
Replace YAML::XS by YAML::Safe.
Fixed many typos and pod markup. Added reference in perlfaq to new ~ syntax in indented here-docs.
Warnings enabled by setting the WARN_ON_ERR flag in $PerlIO::encoding::fallback are now only produced if warnings are enabled with use warnings "utf8"; or setting $^W. [perl #131683]
Allow Off_t smaller than size_t. (Win64 with USE_LARGE_FILES=undef) [perl #133422]
Typos in pod.
podlators 4.12Skip the man/no-encode.t test if Encode is already loaded while running the test suite, which seems to happen sometimes with CPAN Testers tests.
Merge with updates from 3.39 Releaxed some str types, which could be Pod::Simple::LinkSection also.
Use quadmath versions of log10, ldexp and signbit. [perl #133510]
stdio.h on Cygwin doesn't expose cuserid with _GNU_SOURCE
Add SECURITY AND PORTABILITY warning paragraph to pod.
Document -Drv
Fix a test for CORE::dump
Adapt for layers on STDERR, which failed if PERL_UNICODE was set and the current locale was a UTF-8 locale.
Re-order and comment deprecated IDN constants. Old MinGW lacks ADDRESS_FAMILY; use USHORT instead (RT128550). Add PF_LOCAL/AF_LOCAL constants (RT127089). Provide sockaddr_un on WIN32 (RT128550).
Fix and clarify handling of recurs_sv. [perl #133326]
Remove exploit code from the pod, which triggered virus scanners. Note that the perl5 release did that wrong. [perl #133706]
Keep Storable::Limit, in opposition to [perl #133708]. Not my idea, but removing it is even worse. The ticket goal is fine, but the implementation not.
Fix cxt->pseen leak
Enable >2GB AvFILL check on store_hook (64bit) Too many references returned by STORABLE_freeze. It wrapped around previously. Fixes Coverity CID #187854. Move __Storable__.pm into Storable_pm.PL [cperl #374]
Merge with 2.38
Add extra lock tracing via -DSHARED_TRACE_LOCKS
Make utime() available only if we have both fd and name setting [perl #133030]
Adjust Makefile.PL for windows: the DEFINE() in Makefile.PL can't be be in sub init() because that sub isn't called on windows
t/itimer.t: avoid race condition. don't truncate nanosec utime
fallback/const-c.inc: Avoid compiler warning showing up on darwin.
UCET is updated (for Unicode 10.0.0) as Collate/allkeys.txt. The default UCA_Version is 36. Locale/*.pl and CJK/Korean.pm are updated.
U::C::Locale newly supports locale: cu. tailoring Cyrillic YI as BYELORUSSIAN-UKRAINIAN I with DIAERESIS, affected locale: kk added loc_cu.t in t.
Skip redefinition warnings on cperl since 5.27.2 (Hangul in utf8.h)
Preserve old-style casing for %SCRIPTS, i.e. Egyptian_hieroglyphs for Egyptian_Hieroglyphs.
vars.pm no longer disables non-vars strict when checking if strict vars is enabled. [perl #130674]
More test fixes for 5.6
Add ffi
Fix various -Wunused warnings, added () usage croaks. Fixed a -Warray-bounds buffer overflow in LONGPATH, and two -Wmaybe-uninitialized.
Fixed internal leaks
arybase was removed.
Setting $[ to a non-zero value has been deprecated since Perl 5.12 and throws a fatal error as of Perl 5.30. See "Assigning non-zero to $[ is fatal" in perldeprecation
B::Debug is no longer distributed with the core distribution. This was announced with v5.28.0. It remains available on CPAN.
Our patched version of YAML::XS is no longer distributed with the cperl core distribution. It was replaced with YAML::Safe.
We have attempted to update the documentation to reflect the changes listed in this document. If you find any we have missed, send email to perlbug@perl.org.
Additionally, the following selected changes have been made:
prepend "./" to local require calls. Mojolicious and Dancer2 are not called young anymore.
More specific documentation of paragraph mode. [perl #133722].
Added a "COMPILER OPTIMIZATIONS" in perlhacktips section, with LTO, PGO, AutoFDO, prelink and BOLT tips.
The following additions or changes have been made to diagnostic output, including warnings and fatal error messages. For the complete list of diagnostic messages, see perldiag.
Invalid script %s in identifier %s for U+%X, Limited Use scripts are forbidden See "Disallowed Limited Use Scripts".
cperl fatalized the error on non-grapheme string delimiters (ie. illegal utf8) in v5.25.3c already, e.g with qr ̂foobar̂. perl5 did it with v5.29.0 but used 3 error lines instead of just one.
"Use of unassigned code point or non-standalone grapheme for a delimiter" uses now 3 lines: "Use of unassigned code point or non-standalone grapheme for a delimiter is not allowed at -, near" "Use of unassigned code point or non-standalone grapheme for a delimiter is not allowed at -, near" "Execution of - aborted due to compilation errors."
cperl fatalized Limited_Use scripts in identifiers with v5.29.2c, they were valid before and acted like %utf8::EXCLUDED_SCRIPTS, i.e. they needed to be declared. Now they throw the Invalid script %s in identifier %s for U+%X, Limited Use scripts are forbidden error. See "Disallowed Limited Use Scripts". cperl-only.
No autovivification of hash slice anymore
Added note that this warning is only enabled with the lexical syntax warnings category, not -w anymore.
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.32), passed through in regex; marked by <-- HERE in m/%s/
Added note that cperl allows more. In cperl only the combinations with \NpPbBxog{} are illegal.
Stabilized the CoreList updater.
Fixed make regen on cperl with a shared libperl. Need to set LD_LIBRARY_PATH for a fullperl with ext/Config.
Avoid bumping config.sh/config.h mtimes when adding valid libffi targets in configpm. This caused massive miniperl rebuilds, but is only needed for perl-dependent targets much later.
Added #ifdef USE_CPERL and #if PERL_VERSION <28 to the API declaration in embed.fnc, making clear the perl5 and cperl API differences in args, argtypes and API declarations.
regen/embed_lib.pl was extended to handle regen-time branches, and ignore perl5 and older verion branches.
Devel-PPPort embed.fnc can now be the same as the core embed.fnc.
Without DEBUGGING the best -flto Link Time Optimization flag is now added to the default ccflags. See http://perl11.github.io/blog/bolt.html and [cperl #381].
The correct -flto= variant is then used for ldflags and lddlflags also. cldflags contains now also the optimize settings, needed for -flto.
ar is replaced by llvm-ar-$llvmsuffix if needed. See "llvmsuffix" below.
ccflags or cc containing -ULTO does not probe and add -flto flags. This is useful for faster B::C testing. E.g. ./Configure -des -Accflags=-ULTO
ccname is now set to "clang" for clang, not "gcc".
On linux "icc" is also set for ccname, and -O3 is preferred over -O2 in most non-DEBUGGING cases with newer compilers.
llvmversion string added to config, like "6.0.1-9.1 (tags/RELEASE_601/final)", analog to gccversion. With clang both variables are set.
Added llvmsuffix config, for bitcode-enabled ar, ranlib, llvm-profdata.
Added ar --plugin path to the gcc lto_plugin, required for binutils ar with static libs with -flto. Esp. on cygwin. Ditto for ranlib.
Added clang and useshrplib support for the two .autofdo targets. See "AutoFDO" in perlhacktips.
Added two .pgo targets, analog to autofdo. See "PGO" in perlhacktips.
Set archname and the target triple myarchname earlier, right after osname.
With -Uinstallmandirs all four variables installman1dir, installman3dir, installsiteman1dir and installsiteman3dir are set to none, skipping installing all man pages.
Added if ffi.h is being found. If so the incpth, ccflags and libs is extended to include libffi.
With useffi this space-seperated string is filled with the valid and supported :nativeconv() ABI's, in uppercase. Matching HAVE_FFI_* defines in config.h. See "nativeconv(STRING)" in attributes.
Disable destructive hash key modifications during iteration by default. use hashiter; to allow it. [cperl #117]
Fixed the error cases when the double or long double probes returned undef.
Replace sha1sum with sha256sum, add gpg -a -b $pkg.
Detect valgrind/valgrind.h and check RUNNING_ON_VALGRIND for setting destruct_level to 2.
Tests were added and changed to reflect the other additions and changes in this release. Furthermore, these significant changes were made:
All modules can now be tested in parallel. From the remaining exceptions ext/Pod-Html/t was fixed a while ago, and cpan/IO-Zlib/t with this release.
We also removed the remaining exception for files starting with 0. We assumed before that the reason a test file's name begins with a 0 is to order its execution among the tests in its directory. Hence, a directory containing such files should be tested in serial order. But in reality there's no such t/00-setup.t test and if so, it would still be run as very first test. CPAN testing via EUMM and Test::Harness does not obey this rule neither. The only existing exception is dist/ExtUtils-MakeMaker/t. The only module to be fixed was ext/FileCache/t. cperl-only.
t/TEST does not need any temp_needs_dot exceptions anymore. All core modules are now . in @INC clean. The only module to be fixed was cpan/libnet/t. cperl-only.
Thoroughly test paragraph mode, using a new test file, t/io/paragraph_mode.t. [perl #133722].
lib/utf8_heavy.t was split away from lib/utf8.t to test utf8_heavy only functions, mostly SCRIPTS. cperl-only.
Added is_CI detection function to t/TEST.
Honor more SKIP_SLOW_TESTS, esp. with B-C, esp. with -flto. Document all used ENV vars in t/harness.
op/hash-rt85026.t finds now almost ever a collision, and skips the test with usesafehashiter enabled (the default). Such not easily detectable destructive hash iters are now forbidden, thanks to usesafehashiter.
op/chdir.t when pwd is a symlink.
DTrace support has been added by also looking for /sbin/dtrace. Install it with sudo yum install -y dtrace-utils dtrace-utils-devel.
Perl's build and testing process on Mac OS X for -Duseshrplib builds is now compatible with Mac OS X System Integrity Protection (SIP).
SIP prevents binaries in /bin (and a few other places) being passed the DYLD_LIBRARY_PATH environment variable. For our purposes this prevents DYLD_LIBRARY_PATH from being passed to the shell, which prevents that variable being passed to the testing or build process, so running perl couldn't find libcperl.dylib.
To workaround that, the initial build of the perl executable expects to find libcperl.dylib in the build directory, and the library path is then adjusted during installation to point to the installed library.
Macports gcc-mp-9 -O is broken with ext/Config. You need to use a different compiler. [p5-Config #13]
sleep() with warnings enabled for a USE_IMP_SYS build no longer warns about the sleep timeout being too large. [perl #133376]
Support for compiling perl on Windows using Microsoft Visual Studio 2019 (containing Visual C++ 14.2) has been added.
Add DynaLoader fallbacks to dl_find_symbol to mimic the POSIX behaviour with an empty libhandle argument: Search the symbol in all loaded/most shared libraries. This is a convenience platform-compat feature for the ffi.
The new Windows 10 DTrace support has not been tested yet.
The hints now detect newer compilers and sets optimize=-O3 by default, needed for better -flto and inlining support, instead of -O2.
Added /usr/local/include to incpth, which helps probing libffi during configpm. If was only in ccflags via -I.
Fixed wrong clang llvm-ar$llvmsuffix and arflags. BSD ar is fine to use -D.
Improved clang -flto support, set default optimize to -O3. -flto still needs the extra gold linker from a clang/llvm ports package for static archives.
Fix 'for reverse @array' bug on AIX. [perl #133558]
Fixed boot_Win32CORE prototype to skip dllexport, fatal with -flto.
Fixed ar for Win32CORE.a by using the gcc liblto plugin.
Skip probing for -flto on cygwin. This reliably fails to find a proper image-base with that many new symbols, the adress space is just too small: Invalid relocation. Offset 0x.. at address 0x.. doesn't fit into 32 bits
Fixed Errno from processing some protected intrinsics directly, which fail with #error "Never use <avx512vpopcntdqintrin.h>, <avx512ifmaintrin.h> directly; include <immintrin.h> instead." #error "Never use <fmaintrin.h> directly; include <immintrin.h> instead." #error "Never use <xsavesintrin.h> directly; include <x86intrin.h> instead."
The first sizing pass has been eliminated from the regular expression compiler. An extra pass may instead be needed in some cases to count the number of parenthetical capture groups.
Added my_atof3() to support parsing strings which are not necessarily NUL-terminated to an NV. grok_atoUV supports that too now.
It is now forbidden to malloc more than PTRDIFF_T_MAX bytes. Much code (including C optimizers) assumes that all data structures will not be larger than this, so this catches such attempts before overflow happens.
A new function "my_strtod" in perlapi or its synonym, Strtod(), is now availabe with the same signature as the libc strtod(). It provides strotod() equivalent behavior on all platforms, using the best available precision, depending on platform capabilities and Configure options, while handling locale-related issues, such as if the radix character should be a dot or comma.
Enabled USE_REENTRANT_API with threads by default on all platforms. See reentr.c. Previously it was only enabled by default on darwin.
Added two new yet unused COP nextstate-like ops: setstate for the first inlined statement and keepstate for further inlined statements. This is part of the general inliner preparation, which is not yet fully enabled. The inliner is only internally used to create composed roles with different pad indices.
Added a new OP_IS_UNOP_AUXOP(op_type) macro for consistency. And a new HV_FETCH_NO_SV flag to allow storing arbitrary pointers as HeVAL.
Added the new experimental inliner, which can clone optrees and its associated data, and fixup the cloned optree for the new target (types, pad indices). This is not yet enabled by default, only used for role composition, not yet for general function inlining or loop unrolling. API: op_clone_optree, plus several internal functions.
For run-time type-checking this new API was added: arg_check_type_sv
embed.fnc has now seperated #ifdef USE_CPERL and more features from the plain perl5 API, because many perl5 64bit overflow bugs and size limitations arise from a broken API already. The lines are documented as perl5 security risk or perl5 limitation. The parser seperates any cperl from the perl5 API differences already at regen-time, plus the obvious compile-time seperation in the generated headers via the ifdef's.
Added t/run/envsize.t for "Restrict my_setenv to ARG_MAX"
Regular expression matching no longer leaves stale UTF-8 length magic when updating $^R. This could result in length($^R) returning an incorrect value.
Fixed a failure to match properly.
An EXACTFish regnode has a finite length it can hold for the string being matched. If that length is exceeded, a second node is used for the next segment of the string, for as many regnodes as are needed. Care has to be taken where to break the string, in order to deal multi-character folds in Unicode correctly. If we want to break a string at a place which could potentially be in the middle of a multi-character fold, we back off one (or more) characters, leaving a shorter EXACTFish regnode. This backing off mechanism contained an off-by-one error. [perl #133756].
A bare eof call with no previous file handle now returns true. [perl #133721]
Failing to compile a format now aborts compilation. Like other errors in sub-parses this could leave the parser in a strange state, possibly crashing perl if compilation continued. [perl #132158]
If an in-place edit is still in progress during global destruction and the process exit code (as stored in $?) is zero, perl will now treat the in-place edit as successful, replacing the input file with any output produced.
This allows code like:
perl -i -ne 'print "Foo"; last'to replace the input file, while code like:
perl -i -ne 'print "Foo"; die'will not. Partly resolves [perl #133659].
A regression in 5.28 caused the following code to fail
close(STDIN); open(CHILD, "|wc -l")'because the child's stdin would be closed on exec. This has now been fixed.
Fixed an issue where compiling a regexp containing both compile-time and run-time code blocks could lead to trying to compile something which is invalid syntax.
Fixed build failures with -DNO_LOCALE_NUMERIC and -DNO_LOCALE_COLLATE. [perl #133696].
Prevent the tests in ext/B/t/strict.t from being skipped. [perl #133713].
/di nodes ending or beginning in s are now EXACTF. We do not want two EXACTFU to be joined together during optimization, and to form a ss, sS, Ss or SS sequence; they are the only multi-character sequences which may match differently under /ui and /di.
SDBM_File is now more robust with corrupt database files. The improvements do not make SDBM files suitable as an interchange format. [perl #132147]
binmode($fh); or binmode($fh, ':raw'); now properly removes the :utf8 flag from the default :crlf I/O layer on Win32. [perl #133604]
pack "u", "invalid uuencoding" now properly NUL terminates the zero-length SV produced. [perl #132655]
The new in-place editing code no longer leaks directory handles. [perl #133314]
If an in-place edit is still in progress during global destruction and the process exit code (as stored in $?) is zero, perl will now treat the in-place edit as successful, replacing the input file with any output produced.
This allows code like:
perl -i -ne 'print "Foo"; last'to replace the input file, while code like:
perl -i -ne 'print "Foo"; die'will not. Partly resolves [perl #133659].
Warnings produced from constant folding operations on overloaded values no longer produce spurious "Use of uninitialized value" warnings. [perl #132683]
Fix for multiconcat "mutator not seen in (lex = ...) .= ..." [perl #133441]
Detect more "This use of my() in a false conditional" in perldiag cases, esp. scalar assignments like my $x = 0 if 0; [perl #133543]
Regexp script runs were failing to permit ASCII digits in some cases. [perl #133547]
Improve the debugging output for calloc() calls with -Dm. [perl #133439]
On Unix-like systems supporting a platform-specific technique for determining $^X, Perl failed to fall back to the generic technique when the platform-specific one fails (for example, a Linux system with /proc not mounted). This was a regression in Perl 5.28.0. [perl #133573]
The experimental reference aliasing feature was misinterpreting array and hash slice assignment as being localised, e.g.
\(@a[3,5,7]) = \(....);was being interpreted as:
local \(@a[3,5,7]) = \(....);pack() no longer can return malformed UTF-8. It croaks if it would otherwise return a UTF-8 string that contains malformed UTF-8. This protects agains potential security threats. [perl #131642]
See "Any set of digits in the Common script are legal in a script run of another script".
Perl now exposes POSIX getcwd (and on cperl the safer get_current_dir_name) as Internals::getcwd() if available. This is intended for use by Cwd.pm during bootstrapping and may be removed or changed without notice. This fixes some bootstrapping issues while building perl in a directory where some ancestor directory isn't readable. [perl #133951]
Made the gmtime_r and localtime_r fallbacks thread-safe.
Fix valid_ident() for 128-255 chars not to use isIDFIRST_A nor isIDCONT_A, _A does not match them. Fixes t/uni/method.t for chars like oslash
Fix newSVpvn_share to store the HEK_WASUTF8 flag if downgraded. Improve newSVpvn_share to downgrade only once, not twice.
A shared string and a GV with chars in the range 128-255 are downgraded to bytes without the UTF8 flag. Store at least the HEK_WASUTF8 flag then.
Check bless name under use strict 'names'.
The second argument to bless, the new classname, is now checked to be a valid package name identifier when strict 'names' is in effect, and the name is now normalized. [cperl #376]
Fix cperl 5.28.0c regression disallowing calling subnames with a quote in a package qualifier position. [cperl #371]
This is now valid again:
sub don't { 0 } print don't;With v5.28.0c only the declaration sub don't { 0 } was valid, it accepted the subroutine don't, but the call print don't; was invalid, it was looking for a second single-quote string delimiter. Note that in cperl ' is no perl4-style package seperator.
Fixed a Safe double-free of a XS import method in a leaf package. This really is a problem in Safe (see the outcommented code there), but the fix was easiest to do in gp_free.
gp_free in global destruction with Safe may double-free a temporary CV, like &Safe::Root0::strict::import@, i.e. with dist/Storable/t/code.t The fix is similar to the AV double-free protection of @_ in gp_free.
Fixed refs signature args, which dropped their refcnt to 0 when leaving the sub, so use-after-free stack corruption might occur. It also restored by ptr (SV**) not by value (SV*), which was unsafe when the ptr moved. [cperl #395] cperl-only relevant. perl5 uses args in @_, cperl on the stack.
Mu::new arguments (fields) are now checked for arity and type violations. Mu::new is the default cperl constructor for all class objects. See [cperl #387]
E.g.
class Foo { has int $bar };
my $foo = Foo->new( bar => "baz" );
print $foo->bar;fails with 'Too many arguments for method new. Want: 1, but got: 2'. Named arguments are not yet supported.
class Foo { has int $bar };
my $foo = Foo->new( bar => "baz" );
print $foo->bar;warns with 'Type of arg $bar to Foo should be int (not Str)'. cperl-only relevant.
The perl debugger -d mostly does not work with signatures, esp. with Test::More tests. [cperl #360]
The latest msys2 64bit gcc-9.1.0 compiler fails on two tests: op/sort.t and op/signatures.t, similar to the darwin gcc-9 where the optimizer generates wrong code in XSConfig. There are also huge performance regressions with the tree-optimizer. [cperl #397]
Don't use gcc-9.1 on x86_64.
Detect shared-only import libraries like libNAME.dll.a, which have no static counterpart libNAME.a, which happens e.g. for libffi on cygwin or msys.
cperl 5.30.0c represents approximately 10 months of development since cperl 5.28.0c and contains approximately 580,000 lines of changes across 2,400 files from 52 authors.
Excluding auto-generated files, documentation and release tools, there were approximately 410,000 lines of changes to 850 .pm, .t, .c and .h files.
The following people are known to have contributed the improvements that became cperl 5.30.0:
Karl Williamson, Reini Urban, David Mitchell, Tony Cook, James E Keenan, Sisyphus, Steve Hay, Tomasz Konojacki, H.Merijn Brand, Dagfinn Ilmari Mannsåker, Nicolas Rochelemagne, Aaron Crane, Daniel Dragan, Craig A. Berry, Sawyer X, Andy Dougherty, Karen Etheridge, Eugen Konkov, Richard Leach, Chris 'BinGOs' Williams, Unicode Consortium, Aristotle Pagaltzis, Abigail, Niko Tyni, Ryan Voots, Zak B. Elep, Pali, Hugo van der Sanden, Yves Orton, Slaven Rezic, Dan Dedrick, Dominic Hargreaves, François Perrad, Hauke D, Jakub Wilk, Matthias Bethke, Shlomi Fish, Andreas König, James Clarke, Jim Cromie, Brian Greenfield, E. Choroba, John SJ Anderson, Dan Book, Petr Písař, Ed J, Leon Timmermans, David Cantrell, Phil Pearl (Lobbes), Alexandr Savca, Tina Müller, Graham Knop.
The list above is almost certainly incomplete as it is automatically generated from version control history including the perl and cperl repos. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker and the cperl github issues.
Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.
For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.
Generated with:
cperl Porting/acknowledgements.pl cperl-5.28.0..HEAD -cIf you find what you think is a bug, you might check the perl bug database at https://rt.perl.org/ . There may also be information at http://www.perl.org/ , the Perl Home Page.
If you believe you have an unreported bug, please run the cperlbug program included with your release. Be sure to trim your bug down to a tiny but sufficient test case. Your bug report, along with the output of perl -V, will be sent off to perlbug@perl.org to be analysed by the Perl porting team.
If you think it's a cperl specific bug or trust the cperl developers more please file an issue at https://github.com/perl11/cperl/issues.
If the bug you are reporting has security implications which make it inappropriate to send to a publicly archived mailing list, then see "SECURITY VULNERABILITY CONTACT INFORMATION" in perlsec For details of how to report the issue.
The Changes file for an explanation of how to view exhaustive details on what changed.
The INSTALL file for how to build Perl.
The README file for general stuff.
The Artistic and Copying files for copyright information.