• Announcement: NaviServer 5.0.0 Release

    From neumann@neumann@wu-wien.ac.at (gustafn) to comp.lang.tcl on Sat Jul 12 13:10:46 2025
    From Newsgroup: comp.lang.tcl

    Dear all,

    We are glad to announce the final release of NaviServer 5.0.0, marking
    the most significant update in the project’s history with over 70,000
    lines of changes. This release delivers robust security enhancements, modernized infrastructure, and improved scalability. Key highlights
    include:

    Security by Default: HTTP client requests now validate server
    certificates automatically, with configurable trust exceptions and a
    bundled CA root store. A new pluggable authorization framework (ns_auth) enables scriptable request- and user-level access control.
    Modernized Core: Unix domain socket support, case-insensitive
    ns_set, Argon2 password hashing, and streamlined configuration via
    environment variables (ideal for containers). The reverse proxy is now
    built into the core, with new forward proxy capabilities.

    Enhanced Observability and Scalability: Improved logging (dynamic
    paths, rotation, and new "Security" severity), detailed connection
    diagnostics (ns_conn urldict, ns_connchan debug), runtime introspection (ns_http keepalives, ns_info buildinfo), persistent client connections,
    and streaming HTTP client processing.

    Backward Compatibility & Future-Readiness
    While introducing cutting-edge features, NaviServer 5.0.0 maintains
    strong backward compatibility, with deprecated APIs clearly documented
    and modern replacements provided (e.g., ns_urlencode →
    ns_percentencode). The release introduces/extends over 50 Tcl commands
    and subcommands. The upgrade to MPL 1.2 and Tcl 9 support ensures
    long-term sustainability. Over 20 community-contributed modules (like
    nsdbpg, nssmtpd) have been updated in lockstep.

    The following people contributed to this release:

    Alexander Danilov, Andrew Piskorski, Antonio Pisano, Brendan Graves,
    Brian Fenton, Danilo Raynor, David F, David Osborne, Filip Minic, Georg
    Lehner, Gustaf Neumann, Hector Romojaro, Joe Oldak, Oleg Oleinick,
    Priyank Jalu, Thomas Renner, Zoran Vasiljevic

    Resources:

    Source as tar file: https://sourceforge.net/projects/naviserver/files/naviserver/5.0.0/
    GitHub repository: https://github.com/naviserver-project/naviserver Documentation: https://github.com/naviserver-project/naviserver

    This release is a major leap forward in security, flexibility, and maintainability. See below for the detailed changes: Upgrade today!

    The NaviServer Team

    =======================================
    NaviServer 5.0.0, released 2025-07-08
    =======================================

    520 files changed, 78805 insertions(+), 32992 deletions(-)

    New Features:
    -------------

    Security Enhancements

    • Security by Default when NaviServer is used as an HTTP client
    (ns_http and ns_connchan)
    All HTTP client requests now validate the peer server certificate by default.
    ◦ Introduced the -insecure flag to bypass validation in trusted scenarios.
    ◦ Bundled a ca-bundle.crt with well-known root certificates for out-of-the-box verification support.

    ◦ Fine-Grained Validation Management:
    ▪ Configure validation exceptions in the httpclient section to
    allow specific
    certificate issues (e.g., self-signed certs) for defined IP
    ranges.
    ▪ Define a directory for storing rejected certificates, enabling
    later review,
    manual trust decisions, or audit trails.
    ▪ For details, consult the admin-config manual.

    • Certificate Management
    Added new commands:
    ◦ ns_certctl list – List loaded certificates.
    ◦ ns_certctl reload – Reload certificates at runtime.

    • Pluggable Authorization Framework
    ◦ Introduced flexible, scriptable request- and user-level
    authorization:
    ◦ Supports multiple authorization handlers ("authorities") with
    ordered evaluation.
    ◦ Handlers can return continuation codes (TCL_OK, TCL_BREAK,
    TCL_RETURN)
    to control the authorization chain.
    ◦ New Tcl commands:
    ▪ ns_auth request – Evaluate configured request-level auth
    handlers.
    ▪ ns_auth user – Evaluate user-level authorization logic.

    • Improved Reverse Proxy Trust Management
    ◦ Define trusted upstream proxies via CIDR ranges.
    ◦ Configurable handling of X-Forwarded-For chains, including:
    ▪ Skipping non-routable addresses.
    ▪ Alternative resolution strategies.
    ◦ Improves accuracy and security of client IP logging (e.g., in access.log).
    ◦ For implementation details, see commit ab23158ece.

    • Argon2 Password Hashing New command: ns_crypto::argon2
    Provides support for modern, secure password hashing.

    User Interface and HTML Improvements

    • Modernized Default Start Page
    ◦ Redesigned layout with responsive and accessible design.
    ◦ Added modal dialog for password updates.
    ◦ Improved security guidance for publicly exposed installations.
    ◦ Installer now fetches resources from GitHub (previously
    Bitbucket).
    • Dark Mode Support
    ◦ Automatically respects system preferences via the
    prefers-color-scheme media query.
    ◦ Supported across modern versions of Windows, macOS, iOS, and
    Android.
    • Consistent Visual Styling
    ◦ Unified look and feel across:
    ▪ Default start page
    ▪ NaviServer documentation
    ▪ nsstats module interface

    Configuration and File Management

    • Configuration via environment variables
    ◦ Added ability to pull in environment variables with a certain
    prefix as
    configuration variables for configuration files (new command
    ns_configure_variables). This helps configuration significantly
    for
    container-based deployments, where ports etc. have to be adjusted
    on
    standardized configuration files.

    • New File Location Options
    ◦ Unified log path resolution via logdir or serverdir.
    ◦ Moved serverdir definition from fastpath to the top-level server section;
    it now acts as the root for both logdir and page directories.
    ◦ Added bindir parameter for custom binary module and nsproxy worker locations.

    • Improved Log Configuration
    ◦ Relative log file paths are now resolved consistently based on:
    ▪ serverrootproc (for mass virtual hosting), or
    ▪ serverdir.
    ◦ Supports dynamic log file naming (e.g., using Host header) for per-vhost logging.
    ◦ Log rotation now applies across multiple log targets.
    • Improved Section Parameter Updates
    ◦ New -update flag for ns_section, allowing ns_param to overwrite previously
    defined values rather than appending.

    Proxy Features

    • Reverse Proxy Integration
    ◦ The reverse proxy module is now built into the NaviServer core.
    ◦ Leverages improvements in ns_http, including:
    ▪ Persistent connections
    ▪ Streaming request handling
    ◦ New features:
    ▪ -use_target_host_header flag for ::revproxy::upstream to control
    Host header forwarding.
    ▪ Configurable transport mechanism using either ns_http or
    ns_connchan.
    ▪ Support for backend connections via Unix domain sockets.

    • Forward Proxy Implementation
    ◦ Base implementation included, built on the same scalable
    architecture as the reverse proxy.
    ◦ Full support for the HTTP CONNECT method:
    ▪ Enables tunneling HTTPS connections through the proxy.
    ▪ Eliminates the need for insecure plaintext fallbacks.

    Additional Functional Improvements

    • Case Insensitive ns_sets
    ◦ Optionally define ns_set to be case insensitive.
    Many usages are per definition case insensitive (header fields,
    configuration values, SQL attribute names, ...).
    Has correctness and performance impacts.

    • Unix Domain Socket Support
    ◦ Both ns_http and ns_connchan now support connections via Unix
    domain sockets.

    • Multibyte HTML Entity Support
    ◦ ns_striphtml and ns_unquotehtml now correctly decode multibyte
    numeric HTML entities.

    • New Log Severities
    ◦ Security – For security-related events.
    ◦ Deprecated – For usage of deprecated features (e.g., API calls, parameters).

    • TclPro Debugger Support
    ◦ Significantly improved integration with the TclPro debugger.
    ◦ Resolved missing file references and corrected documentation.
    ◦ See the ns_adp_debug manual for usage.

    • Tcl 9 Compatibility
    ◦ Internal updates to support Tcl 9 features and constraints.
    ◦ Support for Tcl command argument counts >2^31.

    • License Upgrade
    ◦ Changed from MPL 1.1 + GPL to MPL 1.2.
    ◦ Announced previously on the mailing list and Tcl forums.

    Tcl API Changes

    - ns_cache
    • New option cachingmode: Accepts full or none to toggle caching
    behavior.

    - ns_configsection
    • New option -filter: Filters variables by unread, defaulted, or
    defaults.

    - ns_configure_variables
    • New command to pull in configuration variables from environment
    variables (for use in configuration files)

    - ns_conn
    • General enhancements:
    ◦ ns_conn host, ns_conn port, ns_conn protocol: Now return
    appropriate
    values for all request types (not limited to forward proxy
    requests).
    • New subcommands:
    ◦ ns_conn target: Returns the complete request target, including
    query parameters,
    from the HTTP start line.
    ◦ ns_conn fragment: Returns the fragment identifier (after #) if
    present.
    ◦ ns_conn urldict: Parses the request URL into a Tcl dictionary for structured access.
    ◦ ns_conn host: Supports an optional default value if the host is
    not determined.
    ◦ ns_conn details: Returns a dictionary with driver-specific
    connection metadata.

    - ns_connchan
    • Security and diagnostics improvements:
    ◦ ns_connchan debug: Set or query the debug level.
    ◦ ns_connchan connect / ns_connchan open: Now accept additional security-related
    options: -cafile, -capath, -cert, -hostname, -insecure, -driver,
    and -unixsocket.
    ◦ ns_connchan status: Added -server option for multi-server introspection.
    ◦ ns_connchan write: Now performs buffered writes.
    • More detailed timeout handling

    - ns_driver
    • info, names, stats, and threads support the -server option to allow multi-server diagnostics.

    - ns_http
    • Connection handling and introspection:
    ◦ Persistent HTTP connections supported via the keepalive option in httpclient configuration.
    ◦ Introduced support for multiple task threads.
    ◦ Streaming response support for incremental data handling (useful
    for large file transfers
    and LLM interaction).
    ◦ Default timeout configuration for requests.
    ◦ New callbacks: response_header_callback and
    response_data_callback.
    ◦ Renamed -donecallback to -done_callback (old name deprecated).
    ◦ Added support for informational HTTP status codes (e.g., 100
    Continue).
    ◦ New subcommands:
    ▪ ns_http keepalives: Displays active persistent connections.
    ▪ ns_http taskthreads: Provides insight into HTTP client threads.
    ◦ Major internal refactoring of tclhttp.c for modularity and maintainability.

    - ns_ictl
    • getmodules: Now supports the -server option for per-server
    introspection.

    - ns_info
    • Added subcommands:
    ◦ argv: Returns the original argument vector.
    ◦ bindir: Returns the binary directory path.
    ◦ buildinfo: Displays build-time config and version info.
    ◦ logdir: Path to the log directory.
    ◦ meminfo: Displays memory statistics (when using tcmalloc).

    - ns_ip
    • Added subcommands:
    ◦ inany: Checks if an IP matches any configured address.
    ◦ properties: Returns metadata about configured IPs.
    ◦ public: Determines if an IP is publicly routable.
    ◦ trusted: Checks if an IP is trusted.
    ◦ valid: Validates IP syntax.
    • These enhancements supersede the need for ns_subnetmatch.

    - ns_issmallint
    • New command to check if a value is a valid small integer (replaces
    the old,
    now deprecated issmallint).

    - ns_logctl
    • New subcommand grep: Searches log files with pattern matching,
    stripping color codes and handling line continuations.

    - ns_register* Enhancements
    • All ns_register_* commands now support the -constraints option,
    enabling context-sensitive behavior.
    • New command: ns_register_auth allows registration of user- and request-level authorization handlers.

    - ns_server
    • New subcommands:
    ◦ authprocs: Lists registered request/user auth procedures.
    ◦ hosts: Lists registered hostnames for the server.
    ◦ logdir: Returns the server log directory.
    ◦ realm: returns or sets the realm of the server
    ◦ serverdir: Returns the base directory for the virtual server
    (-effective returns the resolved runtime path).
    ◦ vhostenabled: Returns a boolean indicating virtual hosting status.
    • ns_server mapped: New option -all returns a dict with handler and
    pool info.

    - ns_set
    • Case-insensitive support:
    ◦ New -nocase flag for ns_set create and related operations.
    ◦ Deprecated the i* subcommands (e.g., iget, ifind) in favor of
    unified interface.

    • Multi-valued key support:
    ◦ -all option retrieves all values for a given key in ns_set and ns_config.

    • New subcommands:
    ◦ format: Pretty-prints the set contents.
    ◦ stats: Returns memory usage statistics.

    • delkey: Now returns a boolean success flag.

    • Internal improvements:
    ◦ Refactored ns_set internals using dense storage for lower memory
    use and better cache locality.
    ◦ Replaced deeply nested switch logic with modular, maintainable
    code.

    - ns_thread
    • Unified thread creation:
    ◦ Use ns_thread create for all thread types.
    ◦ Deprecated begin and begindetached to align with ns_cond,
    ns_mutex,
    and ns_sema usage.

    - ns_urlspace
    • Wildcard matching now supports path-segment-level matches
    (previously
    limited to leaf nodes).
    • Applied improved matching in nscgi to support directory-wide CGI
    mapping.
    • Renamed option -contextfilter to -constraints in ns_urlspace
    set/unset
    to match the new registration API.

    Database Enhancements
    • ns_db info: Returns a dictionary of metadata for a given DB handle.
    • ns_dbpooldescription: Renamed from the previous ns_pooldescription
    for consistent naming.
    • ns_db rowcount: Fixed in this release to return actual row counts.

    New Utilities
    • ns_fseekchars: Efficiently scans a stream for a string (e.g., for multipart/form-data parsing).
    • ns_joinurl: Constructs well-formed URLs from path components.
    • ns_mkdtemp: Creates a unique temporary directory (POSIX-style
    mkdtemp()).
    • ns_parsehtml: Parses HTML fragments into a structured Tcl dict.
    • ns_parsemessage: Parses MIME-style messages (e.g., emails or HTTP headers).
    • ns_percentencode / ns_percentdecode: Replace deprecated ns_urlencode
    and ns_urldecode
    for robust URL-safe encoding/decoding.

    Changes in Core Modules

    nscgi
    • New command: ns_register_cgi for dynamic CGI handler registration (supports -noinherit, -path, etc.).
    • Supports unregistration via ns_unregister_op.
    • New environment variables: SCRIPT_FILENAME, REQUEST_URI.
    • Refactored to support major web apps (e.g., WordPress, Joomla).
    • See commit 36027b70215 for implementation details.

    nscp
    • Integrated with new authorization system (e.g., via nsperm users).
    • New command nscp users: Lists registered users for nscp
    authentication.

    nsperm
    • Acts as a pluggable authorization provider for request/user scopes.
    • New config parameter: allowLoopbackEmptyUser—permits unauthenticated loopback access when enabled.
    • Added support for setting default server start pages and nsstats.

    nsproxy
    • New subcommand: ns_proxy workers—provides detailed runtime info on proxy worker processes.

    Bug Fixes

    Stability and Crash Resolutions

    • Addressed multiple potential crash scenarios:
    ◦ Robust handling of HTTP CONNECT requests.
    ◦ Fixed crashes caused by missing or empty argument lists in
    commands such as ns_filestat,
    ns_sockcallback, and ns_ictl oncleanup.
    ◦ Prevented crash in ns_log when invoked with an empty message.
    ◦ Resolved a 24-year-old bug in Ns_AdjTime() that could lead to
    fatal errors due to
    microsecond overflow in multithreaded environments.
    ◦ Fixed crash in ns_conn copy when operating on empty content.
    ◦ Prevented crash in ns_sema release with invalid semaphore counts.
    ◦ Corrected off-by-one error in ns_adp_bind_args that could access uninitialized Tcl_Obj values.
    ◦ Fixed crash in ns_inet_ntop due to unsafe memory operations with overlapping regions
    (notably on aarch64 with musl).
    ◦ Resolved crash in Ns_SetIUpdateSz() caused by case mismatch in
    header keys with the legacy C API.
    ◦ Fixed crash during nscp startup when the users section was not configured.
    ◦ Prevented crash in debug mode when the Host: header could not be
    mapped to a virtual server
    and the driver was installed locally.
    ◦ Avoided crash when launching nsd with -c and -t options and no
    home parameter defined.
    ◦ Fixed crash during computation of ns_conn location when the
    network driver was not
    globally installed (global installation now recommended).
    ◦ Fixed crash due to self-destructive header replacement when
    ns_conn outputheaders
    are passed via ns_respond ... -headers ...

    Functional Correctness and Logic Fixes:
    • ns_conn status: Fixed issue where updated status codes were silently ignored.
    • ns_conn doneCallback: Ensured this callback is always invoked in
    ns_http.
    • nsv_dict get: Fixed a memory leak in value retrieval.
    • ns_conn peeraddr: Resolved race condition in pipelined requests that
    could yield incorrect peer addresses.
    • ns_cache_eval -force: Fixed race condition that could produce
    obsolete results.
    • ns_sema create: Corrected handling of initial count values (e.g.,
    1000).
    • ns_trim: Fixed spacing logic to trim only leading and trailing whitespace, preserving internal spacing.
    • ns_config -int: Corrected fallback behavior when invalid values are supplied,
    now correctly using documented defaults.
    • ns_sockcallback: Now gracefully handles missing arguments.
    • ns_socknread: Fixed inaccurate results for buffered connections.
    • ns_crypto::aead::encrypt/decrypt: Restored compatibility with
    OpenSSL 1.1.1.
    • Fixed incorrect parsing of encoded backslashes in URLs.
    • Ns_StrTrimRight(): Corrected UTF-8 handling in right-side string trimming.
    • Fixed ns_conn location when running behind a reverse proxy to always returns a value,
    even in broken configurations.

    OpenSSL fixes
    • Added support for detecting and validating OCSP Must-Staple and AIA presence
    in certificates. Without that, NaviServer might crash, when OCSP is
    turned on,
    and NULL values are passed for AIA URLs (letsencrypt)
    • Introduced stable output buffers for send operations to support
    retries
    after SSL_ERROR_WANT_WRITE, preventing connection failures under
    high load.
    • Improved error handling in OpenSSL integration by draining the error stack via DrainErrorStack().

    HTTP Client Fixes
    • ns_http:
    ◦ Reordered initialization in NsInitServer() to ensure submodules
    can access a fully configured server state.

    HTML and ADP Parsing Fixes
    • ADP Parser:
    ◦ Enhanced support for quoted > characters inside attribute values, aligning with modern HTML parsing rules.
    • return-notice handling:
    ◦ Suppressed spurious error messages when fallback ADP templates are processed outside of a full ADP context.
    • ns_striphtml:
    ◦ Fixed long-standing bug where adjacent HTML entities were
    incorrectly decoded;
    only the first entity was processed.

    Logging and Diagnostics Fixes

    • Fixed misleading log output when dynamically changing extended
    headers via
    ns_accesslog extendedheaders ....
    • Improved error messages when port binding fails due to conflicts
    with driver assignment.
    • Fixed off-by-one error in the virtual server port configuration
    logic.
    • Clarified or corrected multiple logging messages across modules for better diagnostics.

    nscgi Module Fixes
    • Fixed file upload failures when uploads were internally spooled by NaviServer —
    previously returned 500 errors.
    • Corrected processing of CGI script exit codes not 0.
    • Fixed hostname and port reporting for SERVER_NAME.

    Database Fixes
    • ns_db rowcount: Fixed regression where the row count logic was not invoked, rendering the command a no-op.

    Build and Compatibility Fixes
    • Suppressed obsolete --enable-threads warning for Tcl versions where thread support is now enabled by default.
    • Fixed compilation failure with glibc 2.38+ due to PTHREAD_STACK_MIN becoming dynamic via sysconf().

    General Cleanups
    • Fixed small memory leak triggered by serverrootproc reset.
    • Numerous minor typo corrections, comment clarifications, and small
    logic cleanups across modules.

    C-Level Infrastructure, C API Enhancements, and Build System

    Core Infrastructure Improvements
    • Socket Layer Enhancements:
    ◦ Added sendErrno field to the Sock structure to improve diagnostics
    and tracking of write errors.
    • Modernized Initialization:
    ◦ Introduced NS_INIT_ONCE() macro for thread-safe one-time initialization,
    replacing legacy double-lock patterns.
    • Data Structure Utilities:
    ◦ Added utility functions Ns_DListSaveString() and
    Ns_DListFreeElements()
    to simplify dynamic list management.
    ◦ Replaced use of legacy Ns_DString functions with standard
    Tcl_DString,
    modernizing internal data handling.
    • Debugging Support:
    ◦ Added NsHexPrint() for hex-dumping of byte sequences, aiding in
    low-level debugging and analysis.
    • Introspection Enhancements:
    ◦ Introduced Ns_TclReturnCodeString() and Ns_ReturnCodeString() to
    convert internal return codes into readable strings, improving log clarity and diagnostics.
    • Code Quality and Performance:
    ◦ Applied extensive internal refactorings to improve performance,
    cache locality,
    and maintainability across multiple subsystems.

    C API Enhancements

    • General:
    ◦ Added typedefs for Ns_AuthorizeRequestProc, Ns_AuthorizeUserProc,
    Ns_UrlSpaceMatchInfo, Ns_DriverConnInfoProc
    ◦ Added enum for Ns_RequestType, Ns_UrlSpaceOp,
    Ns_DriverClientInitArg
    ◦ Added API calls in ns.h: Ns_ConfigFilename, Ns_ConnTarget, Ns_UrlSpaceMatchInfo,
    Ns_ConnServPtr, Ns_DStringAppendSockState, Ns_RegisterFilter2, Ns_TaskQueueLength,
    Ns_TaskQueueName, Ns_TaskQueueRequests, Ns_ObjvTablePrint, Ns_InfoLogPath, Ns_LogPath,
    Ns_ServerLogDir, Ns_ServerRootProcEnabled, Ns_ServerLogGetFd, Ns_ServerLogCloseAll,
    Ns_ServerLogRollAll, Ns_GetServer, Ns_ServerName,
    Ns_SockSetSendErrno,
    Ns_SockGetSendErrno, Ns_SockGetSendRejected, Ns_SockGetSendCount, Ns_SockFlagAdd,
    Ns_SockFlagClear, Ns_SockSendBufsEx, Ns_SockConnectUnix, Ns_SockGetClientSockAddr,
    Ns_SockGetConfiguredSockAddr, Ns_SockaddrPublicIpAddress, Ns_SockaddrTrustedReverseProxy,
    Ns_SockaddrInAny, Ns_SockaddrAddToDictIpProperties, Ns_TclReturnCodeString, Ns_ReturnCodeString,
    Ns_TclInterpServPtr, Ns_LogDeprecatedParameter,
    Ns_RegisterFastUrl2File

    ◦ Extended Ns_Request, Ns_TclCallback
    ◦ Refactored Ns_ConnReturnMoved() and Ns_ConnReturnRedirect() to
    unify redirection logic.

    • Secure Communication and Validation:
    ◦ Added fine-grained certificate validation APIs
    ◦ Improved error stack draining for OpenSSL operations.

    • Extended C API for ns_connchan:
    ◦ Introduced a minimal C-level API for connchan connections,
    enabling finer control
    over lower-level network operations.

    • Build Platform Compatibility:
    ◦ Updated Windows build system to use NS_IMPORT in place of
    deprecated NS_EXTERN.
    ◦ Avoided use of deprecated C functions such as mktemp() to improve portability and security.

    Build System and Tooling
    • Configuration and Sample Support:
    ◦ Replaced hard-coded OS commands (ls, mv, ...) consistently with
    build variables
    ◦ Replaced hard-coded OpenSSL binary references with the $(OPENSSL)
    build variable.
    ◦ Enhanced log path and configuration variable handling to support
    more dynamic setups.

    • Test Infrastructure:
    ◦ Extended regression and compatibility test coverage to validate
    edge cases
    and cross-platform behavior.

    • Build Metadata Introspection:
    ◦ Added support for build-time introspection of environment
    settings,
    including allocator type (e.g., malloc), compiler version, and Tcl
    build info.

    • Optional Deprecation-Free Builds:
    ◦ Introduced build flag NS_NO_DEPRECATED to exclude deprecated
    functions and
    APIs, mirroring Tcl’s TCL_NO_DEPRECATED mechanism.

    • CI/CD Modernization:
    ◦ Upgraded GitHub Actions workflows from version 3 to version 4 for
    improved performance and compatibility.

    • Sample Configuration Files:
    ◦ Improved sample configurations (nsd-config and openacs-config) to
    use environment-specific settings via ns_configure_variables
    ◦ prefer names http and https instead of nsock and nsssl inside
    sample
    configuration files to ease configuration for new users.

    Documentation Updates

    • Conducted a thorough review and restructuring of all documentation:
    ◦ Tcl Command Documentation:
    ▪ Ensured all implemented Tcl commands and their options are fully documented.
    ▪ Removed documentation for obsolete or unimplemented commands.
    ▪ Deprecated commands are no longer advertised in manuals or used
    in examples.
    ▪ Introduced a dedicated section listing deprecated commands.
    ▪ Enabled automated generation of a complete command reference.
    ◦ Test Alignment:
    ▪ Verified that every implemented Tcl command is covered by
    regression tests.
    ▪ Ensured that all documented commands are also implemented and
    tested.

    • Standardized placeholder syntax across the documentation and syntax
    error messages
    (see commit ffbd32774db for details).

    Manual Page Improvements

    • nscgi: Significantly revised documentation and usage examples for
    clarity and completeness.
    • admin-config.man: Added a new section on “Customizing File Locations”.
    • Added and updated sections covering:
    ◦ Basic templating features.
    ◦ Error handling mechanisms.
    ◦ Recently introduced Tcl commands and options.
    Visual and Structural Enhancements
    • Updated diagrams and usage examples related to:
    ◦ Request processing flow.
    ◦ Reverse proxy (revproxy) behavior.
    Additional Improvements
    • Numerous corrections to spelling, grammar, formatting, and internal linking
    throughout the documentation.

    Deprecation Management Infrastructure
    • Introduced compiler warnings for usage of deprecated Ns_DString* functions
    • Added log severity level Deprecated to better surface deprecated
    usage at runtime
    • Documented all deprecated commands in a dedicated section of the
    command reference
    • Deprecated the use of manual double-checked locking for one-time initialization
    replaced by the NS_INIT_ONCE() macro for safer and cleaner
    initialization semantics

    Deprecations

    Tcl-Level Command Deprecations
    • Deprecated the following Tcl commands:
    ◦ ns_set print → replaced by ns_set format
    ◦ ns_checkurl and ns_requestauthorize → replaced by ns_auth request
    ◦ ns_thread begin → replaced by ns_thread create
    ◦ ns_thread begindetached → replaced by ns_thread create -detached
    ◦ ns_event → replaced by ns_cond
    ◦ ns_pooldescription → replaced by ns_dbpooldescription
    ◦ keyldel, keylget, keylkeys, keylset (from TclX) → replaced by
    native dict functionality in Tcl
    ◦ Legacy experimental functions (marked TBD for over 20 years) are
    now deprecated:
    ▪ ns_browsermatch, ns_choosecharset, ns_cookiecharset, ns_formfieldcharset, ns_formvalueput,
    ns_paren, ns_tagelement, ns_tagelementset
    ◦ Deprecated non-namespaced functions:
    ▪ getformdata, issmallint
    ◦ ns_parsetime: now officially deprecated (was internally marked "To
    be removed" for ~15 years)
    ◦ ns_set_precision: deprecated in favor of standard Tcl idioms

    Tcl-Level Option Deprecations
    • Deprecated options:
    ◦ -buffered in ns_connchan write
    ◦ -donecallback in ns_http → replaced by -done_callback (naming consistency)
    ◦ -binary (previously used to indicate Tcl objects with binary data)
    → replaced
    by -data, aligning with conventions where -binary is a boolean
    flag

    C-Level API Deprecations
    • Deprecated or removed C functions:
    ◦ Ns_ObjvFlags() → replaced by Ns_ObjvIndex() for option parsing (removed redundancy)
    ◦ Ns_SockSendBufs2() → replaced by Ns_SockSendBufsEx() which returns
    an additional errorCode
    ◦ Ns_TclInitInterps() → removed (marked as deprecated since 2005)
    ◦ Deprecated internal usage of OpenSSL and Tcl functions that have
    been marked as deprecated upstream
    ◦ All C functions previously marked as deprecated in source comments
    are now
    officially deprecated

    Configuration Parameter Deprecations
    • Global config parameters:
    ◦ logroll: → replaced by logrollonsignal (standardized log rotation behavior on SIGHUP)
    ◦ serverlog: → replaced by systemlog to reduce ambiguity between
    system and per-server logs

    • Section parameter changes:
    ◦ serverdir in the fastpath section is now deprecated → use
    serverdir in the main
    per-server section instead (reflecting its broader usage scope)

    Changes in Extra Modules

    letsencrypt
    • Tcl9 compatibility changes

    nsauthpam
    • Tcl9 compatibility changes
    • License upgrade

    nscoap
    • Tcl9 compatibility changes
    • NaviServer 5 compatibility
    • License upgrade

    nsdbbdb
    • Replaced deprecated Ns_DString operations
    • Support for LMDB

    nsdbi
    • Removed calls to deprecated functions
    • Tcl9 compatibility changes

    nsdbilite
    • Removed calls to deprecated functions

    nsdbimy
    • Removed calls to deprecated functions

    nsdbipg
    • Removed calls to deprecated functions

    nsdbmysql
    • Removed calls to deprecated functions

    nsdbpg
    • Tcl9 compatibility changes
    • NaviServer 5 compatibility
    • Removed calls to deprecated functions
    • License upgrade

    nsdbsqlite
    • Fixed execution of DML queries.
    • Removed calls to deprecated functions
    • Pulled in new upstream version (3.47.2) of qlite3.c

    nsdns
    • Removed calls to deprecated functions
    • Code cleanup

    nsldap
    • Added support for LDAP URIs in pool configuration
    • Removed calls to deprecated functions
    • Removed unneeded compile macro
    • Fixed result setting of "connected" subcommand
    • Refactor nsldap Tcl command into modular subcommands
    • Tcl9 compatibility changes
    • Updated documentation

    nsloopctl
    • Tcl9 compatibility changes
    • Minor code cleanup

    nsmemcache
    • Removed calls to deprecated functions

    nsoracle
    • Added experimental boolean configuration parameter convertEncoding:
    When this Boolean parameter is true, the connection is created with
    OCIEnvNlsCreate() instead of OCIEnvCreate(). OCIEnvNlsCreate was
    introduced with Oracle9 and is an enhanced version of OCIEnvCreate()
    which is used by default (legacy setting) or when ConvertEncoding is
    not true.
    This parameter performs on-the-fly conversions when connecting to a
    Latin1
    encoded database.
    • Removed calls to deprecated functions
    • Modernized configuration (range checks) and use Ns_ReturnCode when appropriate
    • Tcl9 compatibility changes

    nsphp
    • Removed calls to deprecated functions

    nsshell
    • Updated command list for command completion for NaviServer 5
    • Updated most JavaScript libraries

    nssmtpd
    • Simplified configuration (provided defaults were possible)
    • Added support for providing mailserver password via userinfo in
    relayhost
    • Fixed multi-line response parsing
    • Tcl9 compatibility changes
    • NaviServer 5 compatibility
    • Removed calls to deprecated functions
    • Updated documentation
    • License upgrade

    nssnmp
    • Removed calls to deprecated functions

    nsstats
    • More detailed reports:
    ◦ added list of loaded modules per server
    ◦ added list of all registered domain names for a driver
    ◦ added server log dir to per-server information
    • Refactored web UI (adjusted layout for NaviServer 4.99 and 5)
    • Added support for darkmode
    • Added support for nsperm module, when installed
    • Made parsing of http client log file more robust
    • License upgrade

    nssyslogd
    • Removed calls to deprecated functions
    • NaviServer 5 compatibility

    nstk
    • Tcl9 compatibility changes

    nsudp
    • Tcl9 compatibility changes
    • NaviServer 5 compatibility
    • License upgrade

    nswebpush
    • Tcl9 compatibility changes
    • Updated code for newer and more picky web servers
    • Fixed code for aes128gcm

    revproxy
    • moved code to main repository

    websocket
    • NaviServer 5 compatibility
    • Avoid deprecated messages for ns_conn write -buffered
    • Improved documentation
    • More detailed timeout handling
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Harald Oehlmann@wortkarg3@yahoo.com to comp.lang.tcl on Sat Jul 12 16:15:49 2025
    From Newsgroup: comp.lang.tcl

    Am 12.07.2025 um 15:10 schrieb gustafn:
    Dear all,

    We are glad to announce the final release of NaviServer 5.0.0, marking
    the most significant update in the project’s history with over 70,000
    lines of changes. This release delivers robust security enhancements, modernized infrastructure, and improved scalability. Key highlights
    include:
    Best release ever ! Cudos to the great team,
    Harald


    --- Synchronet 3.21a-Linux NewsLink 1.2