3.00.1

New Features

  • Added configuration parameter resourceSamplingLogDir to set the directory for storing resource tracking logs.

  • Added support for the ternary operator ?:.

  • Implemented tensor data form, enhancing data exchange between the system and deep learning frameworks when used with plugins.

  • Added configuration parameter enableRDMA to enable RDMA communication.

  • Added configuration parameter allowTSDBLevel3Compaction to set whether to allow compaction of level 3 files. Added new parameter level to function triggerTSDBCompaction to specify the level at which to trigger file compaction.

  • Added configuration parameter jobLogRetentionTime to set the retention time of query information stored in data node logs.

  • Added configuration parameter batchJobFileRetentionTime to set the retention time for batch and scheduled job results.

  • Added function remoteRunCompatible that operates in the same way as function remoteRun but skips version verification between local and remote databases.

  • Added parameters priority and parallelism to function scheduleJob to set the priority and parallelism for scheduled jobs.

  • Added parameter bom to function saveText to set whether to include a BOM header when exporting files.

  • Added new optimization functions:

    • fminLBFGSB: Minimizes a function using the L-BFGS-B algorithm.

    • fminNCG: Performs unconstrained minimization using the Newton Conjugate Gradient method.

    • fminBFGS: Minimizes a function using the BFGS algorithm.

    • fminSLSQP: Minimizes a function using the Sequential Least Squares Programming method.

    • brute: Minimizes a function within a given range using the brute-force method.

  • Added functions piecewiseLinFit and pwlfPredict to fit a piecewise linear regression function and make predictions based on the regression model.

  • Added functions gaussianKde and gaussianKdePredict to estimate the probability density of a random variable using the Gaussian kernel estimation and make predictions based on the regression model.

  • Added function vectorAR to analyze multivariate time series using a vector auto-regression model.

  • Added function eqPercent to check element-wise equality of two inputs within a specified tolerance level.

  • Added support for primary key storage engine (enabled by setting engine=”PKEY” in function database).

  • Added parameter primaryKey to function createPartitionedTable for specifying the primary key columns.

  • Added parameter indexes to function createPartitionedTable to specify indexes for vector retrieval.

  • Added function removeNode to remove compute nodes from a cluster.

  • Enhanced function addNode for adding multiple nodes simultaneously.

  • Added support for Chimp compression algorithm.

  • Added configuration parameters TSDBSymbolBaseEvictTime and TSDBCachedSymbolBaseCapacity determine when to evict least recently used SYMBOL base entries based on time or capacity limits.

  • Added function clearAllTSDBSymbolBaseCache to clear all cached SYMBOL base entries that are absent from both the cache engine and ongoing transactions.

  • Added support for DELETE JOIN in the SQL DELETE statement and the sqlDelete function.

  • Added function setTableComment for adding comments to DFS tables. Added parameter comment to SQL CREATE statement.

  • Added configuration parameter memLimitOfAllTempResults to set the memory limit for all temporary partitioned tables generated during distributed queries. Added related functions setMemLimitOfAllTempResults and getMemLimitOfAllTempResults to dynamically set and get the configuration value.

  • Added configuration parameter enableInsertStatementForDFSTable to set whether the insert into statement is allowed for DFS tables.

  • insert into statement now supports inserting multiple rows.

Improvements

  • In metaprogramming, you can now specify column alias using a macro.

  • The registerSnapshotEngine function now allows the snapshot engine to be registered on all data nodes for a DFS table, with a new parameter dedupColumnName added for this purpose.

  • Built-in functions with function patterns can serve as function operators.

  • The snapshot parameter of the restore function now defaults to true instead of false.

  • In asynchronous replication, slave clusters no longer require password configuration through the clusterReplicationExecutionPassword parameter. User credentials are now verified using a more secure method.

  • Enhanced TSDB redo log replay performance.

  • The clearAllCache function has been extended to include cache specific to the TSDB storage engine.

  • The version function now provides more comprehensive version details.

  • interval function enhancements:

    • The duration parameter can be specified using trading calendar identifiers.

    • Non-partitioned table queries now support user-defined aggregate functions when interval's duration and step parameters are inconsistent.

  • The flatten function can be applied on nested tuples.

  • The precision of logisticRegression now matches that of its sklearn equivalent.

  • Improved precision for functions: std, stdp, var, varp, skew, and kurtosis.

  • The func parameter of the moving higher-order function now supports functions returning tuples.

  • A new fill parameter has been added to the addMetrics function to specify the NULL filling method.

  • The parseJsonTable function now accepts tables containing BLOB columns.

  • The isDuplicated function now supports BLOB columns.

  • The fmin function parameters have been adjusted.

  • The rowAlign function supports columnar tuples.

  • Output of the rolling function can now have row labels.

  • TSDB databases now support compression of NULL columns during insertion.

  • The createSchema function now restricts adding a database to a single catalog.

  • Reactive state streaming engine enhancements:

    • prev can be applied on array vector columns.

    • Added support for 35 topN stateful functions.

    • keyColumn is now an optional parameter.

    • User-defined functions (UDFs) that return multiple values can now be used as the rvalue of assignment statements within user-defined stateful functions.

    • Improved error messages.

  • The rightTable parameter of createLookupJoinEngine can now be specified using metacode.

  • The metrics parameter of the following streaming engines now accepts constants or vector of constants: createReactiveStateEngine, createAsofJoinEnginecreateLookupJoinEnginecreateEquiJoinEnginecreateLeftSemiJoinEngine.

  • New parameters cachePurgeTimeColumn, cachePurgeInterval and cacheRetentionTime have been added to enableTablePersistence and enableTableShareAndPersistence functions. These parameters specify how data in memory is cleaned for persisted stream tables.

  • Updated the acceptedDelay parameter in createTimeSeriesEngine and createDailyTimeSeriesEngine to allow specifying the delay time for a window to close and trigger.

  • New parameters outputHandler and msgAsTable, which allow passing calculation results to a unary function, have been added to the following streaming engines: createTimeSeriesEngine, createDailyTimeSeriesEngine, createReactiveStateEngine, createDualOwnershipReactiveStateEngine, createNarrowReactiveStateEngine.

  • INNER JOIN and LEFT JOIN support generating runtime filter using the IN predicate.

  • Enhanced query performance when filtering on partitioning columns in combination with the LIMIT clause.

  • Analytic functions with OVER(PARTITION BY xxx) can now be used within CASE WHEN statements.

  • Optimized higher-order functions accumulate and reduce in JIT.

  • JIT now supports handling indexed matrices.

Issues Fixed

  • Fixed parsing errors preventing addition of function views for no-return functions in module/plugin.

  • Fixed ARM server crashes caused by concurrent openBLAS thread operations.

  • Fixed incorrect results returned by the qclp function.

  • Fixed server crashes when the SQL statements within the metacode specified in the obj parameter of the saveText function were incorrectly enclosed with quotes ('').

  • Fixed server crashes when using at together with higher-order functions eachRight/eachLeft.

  • Fixed server crashes when loading excessively large files with the loadText function.

  • Fixed an issue where the dict function were not recognized in class inheritance.

  • Fixed the getUserHardwareUsage function returning unexpected results when provided with dates.

  • Fixed an issue where the “executionSet” field returned by the getSlaveReplicationQueueStatus function was occasionally randomized.

  • Fixed an issue where strings were incorrectly recognized as DATE values and failed to load with the loadText function.

  • Fixed an error when appending array vectors generated by the fixedLengthArrayVector function to a keyed table.

  • Fixed server crashes when the aggs parameter of wj was specified as count(head()).

  • Fixed SQL parsing issue when a statement included user-defined variables in the SELECT clause alongside a GROUP BY clause.

  • Fixed an issue in pj where, with left and right tables sorted by matching columns, only the first matched row from the right table was selected for joining, causing incomplete results.

  • Fixed SQL optimization failure caused by case mismatches between join columns and table columns.

  • Fixed an issue with partition pruning when the IN clause in a WHERE condition was followed by an in-memory table.