perl5222cdelta - what is new for cperl v5.22.2
This document describes perl-only differences between the cperl 5.22.1 release and the cperl 5.22.2 release.
For cperl also see Changes and perlcperl
Harmonize the lengths, counts and sizes for hash and array indices from a broken mix of int/I32 to SSize_t (31/63bit) with silent truncation overflows and unreachable elements. Changed many wrong API sizes.
STRLEN (i.e. Size_t) for string lengths, and I32 for HEK lengths, also used by names.
Array and hash indices and lengths are now SSize_t (signed 31/63 bits), but really should be UV or Size_t. Esp. the hash riter is not I32 anymore, so you can now iterate over all hash keys, but there were many other silent truncations in helper functions.
This allows on 64bit systems 64bit string lengths, and 63bit array and hash sizes, but limits on 32bit systems the max. hash size from 32 to 31bit.
CVSSv2 Severity: 7.3 (AV:N/AC:L/Au:N/C:P/I:P/A:P/E:U/RL:OF/RC:C/CDP:MH/TD:H/CR:M/IR:M/AR:M)
Increased the parser tokenbuffer from 256 to 1024 characters, thus bumping the maximal length of identifiers and parsed names from 251-255 chars to 1200-1023 characters.
Safer INC path. It does not include "." anymore, unless you set the env PERL_USE_UNSAFE_INC=1. (#118)
CVSSv2 Severity: 8.5 (AV:N/AC:M/Au:S/C:C/I:C/A:C)
Hash security: Disable PERL_HASH_SEED_DEBUG with -t or -T (#114)
Better hash table collision resolution strategy: PERL_PERTURB_KEYS_TOP Still a linked list, but move the found element to the top. This disables PERL_HASH_RANDOMIZE_KEYS. It's still a very primitive and slow hash table, but we wait for a complete rewrite until the next major release.
Better support for -DC and -DCv debugging, with new REFCNT statistics. Added CowREFCNT_inc,dec and overflow assertions.
Re-added the PL_sv_objcount interpreter-global variable used to track the total number of Perl objects in the interpreter, which was removed in Perl 5.20. It is beneficial for all scripts which do not create objects in global scope, so we dont have to check all remaining SV's to call a potential DESTROY method.
This broke binary back-compatibility on MULTIPLICITY and ITHREAD builds with cperl-5.22.1.
Minor improvements in the lexer code (toke.c). Added helper functions, LEX_TYPE_*BRACKETS_ON|OFF macros, better DEBUGGING support. regen_perly keeps now useful temp. files.
Added support for static HEK and AV with B::C, with 6-10% memory improvements for compiled code.
Add HEK_STATIC, AvSTATIC, and av_uncow. Allows COW (copy on write) and COG (copy on grow) for strings and arrays with embedders and the compiler.
Support static lexical warnings. Honor op_static with COPs. This helps producing better compiled perlcc code. (#70)
Support new ccflags -DPERL_SV_DEBUG_FIELDS for easier flags bitfields debugging, even without -DDEBUGGING.
Enhanced the warnings message for warnings 'redefine' to print the caller location also. (# 113) Note that this did not violate the documented diagnostics in perldiag.pod, but the tests for Sub::Install and Test::Trap still need to be patched.
Add support for additional pod/perlcdelta.pod in Porting/pod_rules.pl.
Better IPv6 Support:
Probe for I_NETINET6_IN6, I_NETINET_IP, I_NETINET_IP6 defines. Added the i_netinet_ip, i_netinet_ip6, i_netinet6_in6 Config keys. Added IPv6 to Net::Ping.
Backported some fixes from p5p, merged with perl-5.22.2.
Improved performance of aassign: my (...) = (... , ...); by 10-40%.
Improved performance of index() by 50-70%.
Creating *{"_<$filename"} Perl debugger data structures (see "Debugger Internals" in perldebguts) for XSUBs and const subs has been removed. This removed one glob/scalar combo for each unique .c file that XSUBs and const subs came from. On startup (perl -e0) about half a dozen glob/scalar debugger combos were created. Loading XS modules created more glob/scalar combos. These things were created regardless if the perl debugger was being used or not, unlike for pure perl subs, and ignores that the perl debugger can not debug C code.
Make Configure -O behaviour the default
Support reproducible builds via PERL_BUILD_DATE
Fixed umask for mkstemp(3) calls for older glibc versions
With v5.21.0-67-g60f7fc1, perl started setting umask to 0600 before calling mkstemp(3), and then restoring it afterwards. This is wrong as it tells open(2) to strip the owner read and write bits from the given mode before applying it, rather than the intended negation of leaving only those bits in place.
On modern systems which call open(2) with mode 0600 in mkstemp(3), this clears all the created temporary file permissions. However, any systems that use mode 0666 in mkstemp(3) (like ancient versions of glibc) now create a file with permissions 0066, leaving world read and write permission regardless of current umask.
With many cperl-specific improvements, global destruction fixes and PERL_CORE specific test changes.
MS Windows with MSVC is not yet ready, sorry.
A fast and standard JSON en-/decoder. You should not use PP.
A fast and standard profiler. With PERL_CORE specific test and build changes.
The only way to test sv_dump properly.
A fast and standard YAML en-/decoder. You should not use PP. With most compiler warnings fixed, PERL_CORE specific test changes and new options IndentlessMap for YAML.pm compatibility and NonStrict mode to pass the validator tests and make it usable for CPAN.pm.
Needed for the perl debugger with readline.
Skip CORE test with B::Flags XS installed in sitearch.
Remove unneeded pod2*.PL customizations Fix VERSION_FROM in make_ext.pl
Improved error message for no if.
Added bitwise Removed lexical_topic, new max_version check
Numify $fd in OsFHandleOpen()
Added Normalize.xs
Update Unicode::Collate::CJK::Korean
Added limit to set or return the max. size for a queue
Handle $^O sco
Added AmigaOS support.
Added the IPV6_* constants needed for Net::Ping. Include netinet/ip.h, netinet/ip6.h, netinet6/in6.h if valid.
Upstream development stalled at both cpan and p5p. Now also at https://github.com/rurban/net-ping
New Features:
- Handle IPv6 addresses and the AF_INET6 family.
- Added the optional family argument to most methods. valid values: 6, "v6", "ip6", "ipv6", AF_INET6
- new can take now named arguments, a hashref.
- Added the following named arguments to new: gateway host bind retrans pingstring source_verify econnrefused IPV6_USE_MIN_MTU IPV6_RECVPATHMTU IPV6_HOPLIMIT
- Added the wakeonlan method
- Improve argument default handling
- Added missing documentation
uses new Math::Big*, fixing inf/nan. fix bignum::bpi, use bpi() as a class method, not a function.
Only minor changes
Fix blog() for inf/nan, and more. See https://metacpan.org/changes/distribution/Math-BigRat
Tons of changes from 1.999701 See https://metacpan.org/changes/distribution/Math-BigInt
But kept the deprecated Locale::Codes::Country::alias_code function.
Improved test suite (and made some changes to Codes.pm) based on Devel::Cover. Test suite now has 100% coverage.
shmat() with illegal id fails and sets errno
* move __END__ above POD in documentation (plicease)
* Don't clobber $SIG{__DIE__} and $SIG{__WARN__} in open3_run() (avar)
* Fix 'terminate_on_signal' segfaulting (Petya Kohts)
* Fix usage of fcntl(..., F_GETFL, ...) (tonyc)
Fixed gnu_getopt option not 100% identical to GNU getopt [RT #12675]
Fixed parsing valid real numeric values, e.g. --num=.1 [RT #89513]
PAT_FLOAT => "[-+]?(?=[0-9.])[0-9_]*(\.[0-9_]+)?([eE][-+]?[0-9_]+)?";
Tons of fixes from 2.09
Fix quoting if $^X has spaces
Now merged into ExtUtils-MakeMaker-7.10
Use warnings/strict on all modules
cperl only: Fix alignment probe for 64bit and enforce a SIGBUS on intel also (at least with gnu c compatible compilers: gcc, clang, icc) to mimic errors on other strict platforms: sparc, mips, ppc
Allows now 64bit and darwin multiarch to be probed.
Adds now coverage for the previously never tested branch for -UU32_ALIGNMENT_REQUIRED.
Fixes RT #77919
See https://github.com/rurban/digest-md5/tree/intel-align-rt77919
cperl added support for 5.16 binary names See https://github.com/rurban/Devel-PPPort/tree/516gvhv
Added v5.24 and cperl support.
$Carp::MaxArgNums is no longer treated as if it was one larger.
Removed t/00-report-prereqs.dd template.
Generated from ETHER/YAML-Tiny-1.69
Tests no longer print to stderr unnecessarily. This makes core perl builds a little quieter.
The Test::More dependency has been reduced to 0.88 by emulating 'subtest' for those tests that need it.
Instead of erroring on duplicate keys found in a hash (introduced in version 1.63), now we only warn. This fixes an issue in Strawberry Perl (via CPAN::Meta::YAML) when parsing a configuration file.
Handle cperl-specific 'c' version suffices.
Protect internal _isa_version from non-refs that pass ->isa(version)
Added method for getting structured requirements
In fixing preservation of "0.00", some Module => 0 optimizations were lost; this restores those optimizations
CPAN#110443: [PATCH] fix -perldoc option in pod2usage() method. added missing '-'
CPAN#110446: [feature request] -perldocopt for supplying perldoc options added this option; split of string on whitespace is not optimal, but should do the job
Accept YAML with trailing whitespace in header (Maik Hentsche)
Improved Filter documentation, esp. document that __DATA__ is not filtered.
Renamed existing uniq() to uniqstr() Canonicalise undef to {empty string,zero} in uniq{str,num}() Add a new uniq() with more DWIMish semantics around undef
Fix uses of GET magic by the uniq*() family of functions. GET magic is now always invoked exactly once if it exists.
Added List::Util::uniq() and uniqnum()
Fix build on non-C99 compilers
* Avoid divide-by-zero exception if product()'s accumulator is IV zero (RT105415) * Possible fix for SvTEMP issues in first and any/all/none/notall (RT96343)
Encode::MIME::Header
Both decoder and encoder are rewritten by Pali Rohár. Encoder should be now fully compliant of RFC 2047. Decoder is less strict to be able to decode strings generated by old versions of this module. This fixed a couple of related decode('MIME-header') / encode('MIME-header') bugs and tickets.
Encode::Unicode
Fix RT#107043: If no BOM is found, the routine dies. When you decode from UTF-(16|32) without -BE or LE without BOM, Encode now assumes BE accordingly to RFC2781 and the Unicode Standard version 8.0
Precompile 1252 table as that is now the Pod::Simple default
Fixed compilation under -pedantic
Compilation cleanup, remove hrt_ualarm, hrt_ualarm_itimer
new constants: CLOCK_BOOTTIME CLOCK_HIGHRES CLOCK_MONOTONIC_COARSE CLOCK_MONOTONIC_PRECISE CLOCK_MONOTONIC_RAW CLOCK_REALTIME_COARSE CLOCK_REALTIME_FAST CLOCK_REALTIME_PRECISE CLOCK_SECOND [RT #111170]
clock_getres(), clock_gettime(), clock_nanosleep() emulation for OS X [RT #78566]
Add new optional flags argument to retrieve()/thaw() for flags, or set new global $Storable::flags: 2 for BLESS_OK | 4 TIE_OK. default 6. With 0 disable blessing and tieing of objects for proper security.
Compute CAN_FLOCK at compile-time, to avoid Config in perlcc compiled binaries.
Broke support for restoring restricted hashes [perl #73972] with the new default PERL_PERTURB_KEYS_TOP strategy.
Favour the new builtin XS YAML and JSON variants over PP. Improve the validator tests.
Favour the new builtin XS YAML and JSON variants over PP.
Fixed fatal typos, moved from cpan to dist.
Fixed all 'c' suffix version checks.
Handle 'c' suffix version checks.
cperl accepts versions with and without c, and should not warn about not existing, builtin modules (XSLoader, DynaLoader, strict, attributes) or unparsable versions. [cperl #104]
* Accept the c suffix * perl5 should fail with a c version. * A requirement of a c version can only be fulfilled by cperl (currently).
Fix prerequisite and version checks, Allow cperl c suffices, Honor cperl builtin modules
Protect our cperl changes and non-upstream improvements:
Fixed prereq check for cperl builtins, 'c' version suffix support, make -s silencing, fixed wrong Config dependencies, PERL_USE_UNSAFE_INC=1, call darwin dsymutil
Config is now a XS module, synced with CPAN as XSConfig. The Config tables are now readonly, and many more memory improvements.
Support gv(cvname). Do not print gv(IV \"$$"), the prototype as ref, rather the name of the cvref. [cperl #96]
Avoid uninit hr->name warning.
Stabilize broken t/search50.t search test
Protect ithread_free from deleted PL_modglobal during global destruction.
Optimize boot_DynaLoader, fix a couple of stack errors.
Fix 2 tests broken with readonly Config.
To improve parallel builds on Windows.
Support utf8-encoded XS modulenames. Fixed load_file 2nd arg. Improve load_file errmsg, print the missing sofile. Protect from underflow: use a signed i counter, which can underflow Fixed memory corruption caused by a negative SvCUR. GH #66 Use labs instead of abs on an 64bit IV value.
Beware: untested and platform incompatible.
Fixed typo @shared_objects. It is named @dl_shared_objects. This broke perlcc --staticxs (#71)
Set the IOK bit for dl_expand, caught by the compiler.
Fix B::UNOP_AUX::aux_list design with threads. No 2nd cv arg, return the PADOFFSET not the SV. (#76)
Export SVt_ types. (#80)
Fixed Safe version 'c' problem. Do not check the XS_VERSION of Opcode, because it has a cperl 'c' suffix, which is not yet fully supported in the version check code.
Add a scmp method for string comparison, needed to destinguish between 0.2 and 0.2c but only enforced if the version object is stringified. Using eq is not enough.
fix c suffix comparisons: stringify v0.2c to v0.2c, 0.2c to 0.2c numify 0.2c to 0.002 [0 200], same as 0.2
Lower memory usage
Fixed shmat with negative id. The negative argument will be interpreted as a very large unsigned value. Coverity CWE-687
Avoid div by zero [coverity] Note: fixed differently upstream, with much more branches than here.
static const some local tables.
This modules are currently not ready to be updated.
1.71 breaks t/porting/podcheck.t [RT #113846]
broken for -Dfortify_inc, no "." in @INC
Broken new "_" logic when merged with our 'c' suffix logic.
A new build option USE_NO_REGISTRY has been added to the makefiles. This option is off by default, meaning the default is to do Windows registry lookups. This option stops Perl from looking inside the registry for anything. For what values are looked up in the registry see perlwin32. Internally, in C, the name of this option is WIN32_NO_REGISTRY.
The behavior of Perl using HKEY_CURRENT_USER\Software\Perl and HKEY_LOCAL_MACHINE\Software\Perl to lookup certain values, including %ENV vars starting with PERL has changed. Previously, the 2 keys were checked for entries at all times through Perl processes life time even if they did not exist. For performance reasons, now, if the root key (i.e. HKEY_CURRENT_USER\Software\Perl or HKEY_LOCAL_MACHINE\Software\Perl) does not exist at process start time, it will not be checked again for %ENV override entries for the remainder of the Perl processes life. This more closely matches Unix behaviour in that the enviroment is copied or inherited on startup and changing the variable in the parent process or another process or editing <.bashrc> will not change the enviromental variable in other existing, running, processes.
Windows support was added. There is also new support for parallel builds. The windows smokes are done via appveyor, on MSVC 10 and MSVC 12, on 32bit and 64bit, in optimized Release mode only. No mingw/gcc builds yet.
One glob fetch was removed for each -X or stat call whether done from Perl code or internally from Perl's C code. The glob being looked up was ${^WIN32_SLOPPY_STAT} which is a special variable. This makes -X and stat slightly faster.
During Miniperl's process startup, during the build process, 4 to 8 IO calls related to the process starting .pl and the buildcustomize.pl file were removed from the code opening and executing the first 1 or 2 .pl files.
Avoid access to uninitialized memory in win32 crypt().
Previously the Win32 crypt implementation() would access the first and second characters of the salt, even if the salt was zero length
Fixed global destruction with threads on darwin.
Darwin 11.0 (Lion, July 2011) and newer is no multiarch. There's no PPC support anymore. Use no env MACOSX_DEPLOYMENT_TARGET=10.6 (Snow Leopard) for ld.
This fixes the darwin byteorder to be probed at compile-time, and the d_u32align probe on newer x86_64 darwins.
Enforce SIGBUS when accessing unaligned words with d_u32align / U32_ALIGNMENT_REQUIRED even on Intel CPUs by setting the AC (alignment check) EFLAGS.
Fixed the broken d_u32align probe in Configure and Digest::MD5, broken for darwin and Intel 64bit.
Added a Porting/release_cperl.pod release guide for various platforms, debian, rpm, bsd, osx, windows packaging.
Improved support for threaded builds.
Better DynaLoader support for os2, symbian, Netware.
Fix CVE-2016-2381 sanitize duplicate environ entries
Make sure we use the same value in %ENV that getenv() returns. Remove duplicate entries from environ.
Previously if there was a duplicate definition for a name in environ[] setting that name in %ENV could result in an unsafe value being passed to a child process.
Fix CVE-2012-3878 [RT#776270] module loading security weakness
Treat invalid names to require, such as ::foo as error. New errors:
Bareword in require contains "%s", with "\0", "./" or "../" as invalid substrings.
Bareword in require maps to empty filename
Bareword in require maps to disallowed filename "%s", with absolute filenames, filenames starting with "." or not ending with ".pm".
uncow on utf8::encode [cperl #50]
Fixed double-free with overlong names, esp. Moose [cperl #128]
Improve and fix CopFILE_set in S_incline. Don't call gv_fetchfile_flags 2x with unthreaded perls. Enlarge the temp. buffer on the stack from 128 to 160.
Fixed compilation with -DDEBUGGING and usethreads
In the new loop bounds check elimination code.
Fixed ByteLoader [cperl #75]
Fixed breakage from perl5.22.0 in op.c:newPROG()
Fixed ck_pad: PADSV -> CONST corrupts OpSLOT
Wrong optimization of a readonly lexical to a const did corrupt memory. [cperl #108]
dump fix SEGV on DESTROY SvSTASH [cperl #59]
We can only access a SvSTASH if it is a SvOBJECT. This failed e.g. for %version:: stashes.
Unsure of we should follow p5p recent change to move the DESTROY cache back to mro.
dump fix assert on SvTAIL and !SvSCREAM
It failed with DEBUGGING and ITHREADS on an SCREAM,FAKE,OUR padname, i.e. our @Data_Stack from Test::More. Unfortunately we cannot reproduce this error with core methods, only with Internals::DumpArenas, which dumps the PADNAME as full SV. [cperl #59] [cpan #81635]
podcheck.t: no memory exhaustion
Skip slurping overlarge logfiles > 1MB, e.g. valgrind logfiles, esp. on 32bit. podcheck does not look at the filename extension if it's a potential podfile.
Fixed out-of-bounds access in dirp_dup()
From perl-5.23.7
S_incline avoid overrunning end of parse buffer
[perl #127334] If the rest of the allocation up to the end addressable memory was non-spaces, this loop could cause a segmentation fault.
From perl-5.23.7
[perl #126481] panic for !! with syntax error in /(?[...])/
[perl #126325] don't read past the end of the source for pack [Hh]
With a utf8 target but a non-utf8 source, pack Hh would read past the end of the source when given a length, due to an incorrect condition.
Enforce strict 'subs' in multideref optimisation
The code that checks constant keys and turns them into HEKs swallowed the OP_CONST before the strictness checker could get to it, thus allowing barewords when they should not be
Configure: fix startperl and perlpath
Config entries for cperl.
Enhanced Porting/sync-with-cpan
Support --all updating all cpan/dist modules
Add options -n|--dryrun, -m|--make-c
Handle dist also.
Daniel Dragan took over XSConfig development on CPAN.
Dave Mitchell started to work on some tasks proposed in cperl.
cperl 5.22.2 represents approximately 7 months of development since cperl 5.22.1c and contains approximately 260,000 lines of changes across 1,500 files from 39 authors.
Excluding auto-generated files, documentation and release tools, there were approximately 180,000 lines of changes to 1,000 .pm, .t, .c and .h files.
The following people are known to have contributed the improvements that became cperl 5.22.2:
Reini Urban, Daniel Dragan, Steve Hay, Jarkko Hietaniemi, Tony Cook, Karl Williamson, David Mitchell, Ricardo Signes, Matthew Horsfall, Abigail, Aristotle Pagaltzis, Aaron Crane, David Golden, Jan Dubois, Craig A. Berry, Nicolas Rochelemagne, Todd Rinaldo, James E Keenan, Peter Martini, Nicholas Clark, Niko Tyni, Karen Etheridge, Sisyphus, Victor Adam, Sawyer X, Yves Orton, Stevan Little, Vladimir Timofeev, Andreas König, Leon Timmermans, Hugo van der Sanden, Andy Broad, Father Chrysostomos, Helmut Wollmersdorfer, Lukas Mai, H.Merijn Brand, Herbert Breunung, Ivan Pozdeev, Dagfinn Ilmari Mannsåker.
The list above is almost certainly incomplete as it is automatically generated from version control history including the perl and cperl repos. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker and the cperl github issues.
Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish. Their quality of work succeeds the work of the perl5 porters by far.
If you find what you think is a bug, you might check the articles recently posted to the comp.lang.perl.misc newsgroup and the perl bug database at https://rt.perl.org/ . There may also be information at http://www.perl.org/ , the Perl Home Page.
If you believe you have an unreported bug, please run the perlbug program included with your release. Be sure to trim your bug down to a tiny but sufficient test case. Your bug report, along with the output of perl -V, will be sent off to perlbug@perl.org to be analysed by the Perl porting team.
If you think it's a cperl specific bug or trust the cperl developers more please file an issue at https://github.com/perl11/cperl/issues.
If the bug you are reporting has security implications, which make it inappropriate to send to a publicly archived mailing list, then please send it to perl5-security-report@perl.org. This points to a closed subscription unarchived mailing list, which includes all the p5p core committers, who will be able to help assess the impact of issues, figure out a resolution, and help co-ordinate the release of patches to mitigate or fix the problem across all platforms on which Perl is supported. Please only use this address for security issues in the Perl core, not for modules independently distributed on CPAN.
If you trust the cperl developers more, please send an email to them. The p5p security teams skips many security issues, or are unwilling to fix them.
The Changes file for an explanation of how to view exhaustive details on what changed.
The INSTALL file for how to build Perl.
The README file for general stuff.
The Artistic and Copying files for copyright information.