[Feature][Enhancement] Enhance patterns command with additional sample_logs output field #4155
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Songkan Tang <[email protected]>
Signed-off-by: Songkan Tang <[email protected]>
songkant-aws
requested review from
ps48,
kavithacm,
derek-ho,
joshuali925,
dai-chen,
YANG-DB,
mengweieric,
vamsimanohar,
Swiddis,
penghuo,
seankao-az,
MaxKsyunz,
Yury-Fridlyand,
anirudha,
forestmvey,
acarbonetto,
GumpacG,
ykmr1224,
LantaoJin,
noCharger,
qianheng-aws and
yuancu
as code owners
Signed-off-by: Songkan Tang <[email protected]>
yuancu
approved these changes
Sep 24, 2025
qianheng-aws
approved these changes
Sep 24, 2025
joshuali925
pushed a commit
that referenced
this pull request
Sep 24, 2025
* Doc enhancement for eventstats and bin command (#4117) * distinct_count doc for eventstats Signed-off-by: Kai Huang <[email protected]> * doc enhancement Signed-off-by: Kai Huang <[email protected]> * add fields for consistency between different Java versions Signed-off-by: Kai Huang <[email protected]> * remove changes Signed-off-by: Kai Huang <[email protected]> * add bin to index.rst Signed-off-by: Kai Huang <[email protected]> * add link Signed-off-by: Kai Huang <[email protected]> * fix Signed-off-by: Kai Huang <[email protected]> --------- Signed-off-by: Kai Huang <[email protected]> * Implement `Append` command with Calcite (#4123) * Implement Append Command Signed-off-by: Songkan Tang <[email protected]> * Fix spotless check Signed-off-by: Songkan Tang <[email protected]> * Rephrase append.rst Signed-off-by: Songkan Tang <[email protected]> * Support subsearch different index for append command Signed-off-by: Songkan Tang <[email protected]> * Fix some tests and add cross cluster IT Signed-off-by: Songkan Tang <[email protected]> * Not support empty subsearch input for now Signed-off-by: Songkan Tang <[email protected]> * Fix doctest Signed-off-by: Songkan Tang <[email protected]> * Support empty source edge case Signed-off-by: Songkan Tang <[email protected]> * Fix anonymizer tests Signed-off-by: Songkan Tang <[email protected]> * Add missing test cases for nested join or lookup command in appended subsearch Signed-off-by: Songkan Tang <[email protected]> * Fix compile issue Signed-off-by: Songkan Tang <[email protected]> --------- Signed-off-by: Songkan Tang <[email protected]> * `Bin` command big5 queries (#4163) * Bin command big5 queries Signed-off-by: Kai Huang <[email protected]> * update IT Signed-off-by: Kai Huang <[email protected]> * fix Signed-off-by: Kai Huang <[email protected]> * remove tests Signed-off-by: Kai Huang <[email protected]> --------- Signed-off-by: Kai Huang <[email protected]> Signed-off-by: Kai Huang <[email protected]> * Don't recreate indices on every test (#4222) Signed-off-by: Simeon Widdis <[email protected]> * Enable pushdown optimization for filtered aggregation (#4213) * Enable filtered aggregation pushdown Signed-off-by: Chen Dai <[email protected]> * Add basic UT and ignore IT for now Signed-off-by: Chen Dai <[email protected]> * Enable aggregate case to filter rule and fix UT and IT Signed-off-by: Chen Dai <[email protected]> * Add expected json file for no pushdown test Signed-off-by: Chen Dai <[email protected]> * Remove unnecessary aggregate case to filter rule Signed-off-by: Chen Dai <[email protected]> * Add UT for IS_TRUE support Signed-off-by: Chen Dai <[email protected]> * Add more explain IT Signed-off-by: Chen Dai <[email protected]> * Refactor UT Signed-off-by: Chen Dai <[email protected]> * Extract aggregate filter analyzer abstraction Signed-off-by: Chen Dai <[email protected]> * Add more UT Signed-off-by: Chen Dai <[email protected]> * Refactor UT with fluent API Signed-off-by: Chen Dai <[email protected]> * Add UT for distinct count Signed-off-by: Chen Dai <[email protected]> * Address comment by adding UT for script filter pushdown Signed-off-by: Chen Dai <[email protected]> * Fix spotless Signed-off-by: Chen Dai <[email protected]> --------- Signed-off-by: Chen Dai <[email protected]> * Split up our test actions into unit, integ, and doctest. (#4193) * Run unit test suites in parallel Signed-off-by: Simeon Widdis <[email protected]> * Split out our test actions Signed-off-by: Simeon Widdis <[email protected]> * Make unit test step run in parallel Signed-off-by: Simeon Widdis <[email protected]> * Fix removed bwc tests Signed-off-by: Simeon Widdis <[email protected]> * Add another missing parallel flag Signed-off-by: Simeon Widdis <[email protected]> --------- Signed-off-by: Simeon Widdis <[email protected]> Signed-off-by: Simeon Widdis <[email protected]> * [Feature] Core Implementation of `rex` Command In PPL (#4109) * rex - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * stop using utils Signed-off-by: Jialiang Liang <[email protected]> * fix spotless check Signed-off-by: Jialiang Liang <[email protected]> * offset_field - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * max_match - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * sed - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * fix name capture group for extraction Signed-off-by: Jialiang Liang <[email protected]> * add rex rst doc Signed-off-by: Jialiang Liang <[email protected]> * IT - initial setup Signed-off-by: Jialiang Liang <[email protected]> * add a analyzer test for legacy engine Signed-off-by: Jialiang Liang <[email protected]> * Add UT for rex Signed-off-by: Jialiang Liang <[email protected]> * sed - add pushdown for sed and explain IT and IT with fix Signed-off-by: Jialiang Liang <[email protected]> * anonymizer - add rex for anonymizer and test Signed-off-by: Jialiang Liang <[email protected]> * Add cross cluster IT for rex Signed-off-by: Jialiang Liang <[email protected]> * peng - resolve comments for rst doc 0 Signed-off-by: Jialiang Liang <[email protected]> * peng - address some comments 1 Signed-off-by: Jialiang Liang <[email protected]> * peng - resolve comment in rst doc to add a java doc link Signed-off-by: Jialiang Liang <[email protected]> * kai - modify the bin ast builder test Signed-off-by: Jialiang Liang <[email protected]> * peng - fix the extraction behavior without filter even when there is zero match Signed-off-by: Jialiang Liang <[email protected]> * fix rex explain no pushdown Signed-off-by: Jialiang Liang <[email protected]> * change the offset val output format Signed-off-by: Jialiang Liang <[email protected]> * fix rst file Signed-off-by: Jialiang Liang <[email protected]> * peng - SWITCH TO USE CALCITE NATIVE OPERATORS Signed-off-by: Jialiang Liang <[email protected]> * Peng - fix tests after operator change Signed-off-by: Jialiang Liang <[email protected]> * support mode=extract and update doc Signed-off-by: Jialiang Liang <[email protected]> * fix the issue after rebase Signed-off-by: Jialiang Liang <[email protected]> * peng - enforce specifying field in antlr for now Signed-off-by: Jialiang Liang <[email protected]> * relocate rex cmd IT Signed-off-by: Jialiang Liang <[email protected]> * peng - simplify vistFunciton Signed-off-by: Jialiang Liang <[email protected]> * peng - add UT for RexExtractMultiFunction Signed-off-by: Jialiang Liang <[email protected]> * peng - add UT RexOffsetFunction Signed-off-by: Jialiang Liang <[email protected]> * fix some tests Signed-off-by: Jialiang Liang <[email protected]> * DECOUPLE SED + OFFSET FIELD Signed-off-by: Jialiang Liang <[email protected]> * Improve error handling for extract Signed-off-by: Jialiang Liang <[email protected]> * add this rex rst into index Signed-off-by: Jialiang Liang <[email protected]> * fix return type in extract multi Signed-off-by: Jialiang Liang <[email protected]> * add rex doc into doc test Signed-off-by: Jialiang Liang <[email protected]> * fix doc test Signed-off-by: Jialiang Liang <[email protected]> * Fix linting Signed-off-by: Jialiang Liang <[email protected]> * fix rebase issue Signed-off-by: Jialiang Liang <[email protected]> * fix regex anonymizer tests Signed-off-by: Jialiang Liang <[email protected]> * fix analyzer test and setup to use util function Signed-off-by: Jialiang Liang <[email protected]> * lint fix Signed-off-by: Jialiang Liang <[email protected]> * fix doc test Signed-off-by: Jialiang Liang <[email protected]> * Add max match limit implementation Signed-off-by: Jialiang Liang <[email protected]> * fix anonymizer test Signed-off-by: Jialiang Liang <[email protected]> * peng - simplify if Signed-off-by: Jialiang Liang <[email protected]> * peng - make extract multi to only handle the case of max_match > 1 Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]> * Add wildcard support for rename command (#4019) * add wildcard support for rename Signed-off-by: Ritvi Bhatt <[email protected]> * fix calcite wildcard support and add tests Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> * add check to analyzer Signed-off-by: Ritvi Bhatt <[email protected]> * update doc formatting Signed-off-by: Ritvi Bhatt <[email protected]> * remove v2 engine wildcard support Signed-off-by: Ritvi Bhatt <[email protected]> * update doc Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> * support cascading rename Signed-off-by: Ritvi Bhatt <[email protected]> * update formatting Signed-off-by: Ritvi Bhatt <[email protected]> * add cross cluster test Signed-off-by: Ritvi Bhatt <[email protected]> * add test for cascading rename Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> * add test for cascading rename Signed-off-by: Ritvi Bhatt <[email protected]> * change behavior for renaming existing fields Signed-off-by: Ritvi Bhatt <[email protected]> * add tests and update docs Signed-off-by: Ritvi Bhatt <[email protected]> * update docs Signed-off-by: Ritvi Bhatt <[email protected]> * update docs Signed-off-by: Ritvi Bhatt <[email protected]> * fix renaming to same name Signed-off-by: Ritvi Bhatt <[email protected]> * fix behavior for consecutive wildcards/address comments Signed-off-by: Ritvi Bhatt <[email protected]> * add back import Signed-off-by: Ritvi Bhatt <[email protected]> * fix doc Signed-off-by: Ritvi Bhatt <[email protected]> * fix doc Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> --------- Signed-off-by: Ritvi Bhatt <[email protected]> Signed-off-by: ritvibhatt <[email protected]> * Add support for `median(<value>)` (#4234) * First revision Signed-off-by: Aaron Alvarez <[email protected]> * Fixing documentation Signed-off-by: Aaron Alvarez <[email protected]> * Removing unnecessary comments Signed-off-by: Aaron Alvarez <[email protected]> * Fixinf stats.rst documentation Signed-off-by: Aaron Alvarez <[email protected]> * Fixing documentation Signed-off-by: Aaron Alvarez <[email protected]> * Addressing comments Signed-off-by: Aaron Alvarez <[email protected]> --------- Signed-off-by: Aaron Alvarez <[email protected]> Signed-off-by: Aaron Alvarez <[email protected]> Co-authored-by: Aaron Alvarez <[email protected]> * Dynamic source selector (#4116) Signed-off-by: Vamsi Manohar <[email protected]> * Add gitignore (#4258) Signed-off-by: Tomoyuki Morita <[email protected]> * Support join field list and join options (#3803) * Support join field list and join options Signed-off-by: Lantao Jin <[email protected]> * Add SPL-compatible syntax setting Signed-off-by: Lantao Jin <[email protected]> * Revert SPL settings Signed-off-by: Lantao Jin <[email protected]> * Fix IT Signed-off-by: Lantao Jin <[email protected]> * Fix IT Signed-off-by: Lantao Jin <[email protected]> * Support max=n option Signed-off-by: Lantao Jin <[email protected]> * support max=n in sql-like join syntax Signed-off-by: Lantao Jin <[email protected]> * Add Explain IT for new join syntax Signed-off-by: Lantao Jin <[email protected]> * Refactor the user doc Signed-off-by: Lantao Jin <[email protected]> * Fix conflicts Signed-off-by: Lantao Jin <[email protected]> * Fix conflicts Signed-off-by: Lantao Jin <[email protected]> * Disable the collapse pushdown Signed-off-by: Lantao Jin <[email protected]> * refactor Signed-off-by: Lantao Jin <[email protected]> * Fix IT Signed-off-by: Lantao Jin <[email protected]> --------- Signed-off-by: Lantao Jin <[email protected]> * Support first/last aggregate functions for PPL (#4223) * Support first/last aggregation functions for PPL Signed-off-by: Kai Huang <[email protected]> * Support null Signed-off-by: Kai Huang <[email protected]> * remove legacy Signed-off-by: Kai Huang <[email protected]> * update doc Signed-off-by: Kai Huang <[email protected]> * fix doctest Signed-off-by: Kai Huang <[email protected]> * fix stats.rst file Signed-off-by: Kai Huang <[email protected]> * fixes Signed-off-by: Kai Huang <[email protected]> * move pushdown logic to AggregateAnalyzer Signed-off-by: Kai Huang <[email protected]> * fix IT and update null handling Signed-off-by: Kai Huang <[email protected]> * add test cases for null handling Signed-off-by: Kai Huang <[email protected]> * handle parallelism Signed-off-by: Kai Huang <[email protected]> * Simplify CalciteExplainIT and add UT for AggregateAnalyzer Signed-off-by: Kai Huang <[email protected]> # Conflicts: # opensearch/src/test/java/org/opensearch/sql/opensearch/request/AggregateAnalyzerTest.java * fixes Signed-off-by: Kai Huang <[email protected]> --------- Signed-off-by: Kai Huang <[email protected]> * Fix gitignore to ignore symbolic link (#4263) add comment Signed-off-by: Tomoyuki Morita <[email protected]> * Push down limit operator into aggregation bucket size (#4228) * Push down limit operator into aggregation bucket size Signed-off-by: Heng Qian <[email protected]> * Fix IT Signed-off-by: Heng Qian <[email protected]> * Fix robust issue in OpenSearchLimitIndexScanRule Signed-off-by: Heng Qian <[email protected]> * Refine comments Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Heng Qian <[email protected]> * Fix the IT issue caused by merging conflict (#4270) Signed-off-by: Heng Qian <[email protected]> * Print links to test logs after integTest (#4273) * Print links to test logs after integTest Signed-off-by: Tomoyuki Morita <[email protected]> * print even when tets failed Signed-off-by: Tomoyuki Morita <[email protected]> --------- Signed-off-by: Tomoyuki Morita <[email protected]> * [Feature] Implementation of mode `sed` and `offset_field` in rex PPL command (#4241) * [Feature] Implementation of mode sed and offset_field in rex PPL command Signed-off-by: Jialiang Liang <[email protected]> * update rex rst doc Signed-off-by: Jialiang Liang <[email protected]> * chen - address comment and merge grammar in parser Signed-off-by: Jialiang Liang <[email protected]> * chen - limit offset field only in extraction mode Signed-off-by: Jialiang Liang <[email protected]> * chen - specify exception type of o_f UDF Signed-off-by: Jialiang Liang <[email protected]> * chen - add exception type of o_f UDF - 2 Signed-off-by: Jialiang Liang <[email protected]> * chen - add exception type of o_f UDF - also fix the test Signed-off-by: Jialiang Liang <[email protected]> * chen - alphabetical order of o_f return Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]> * Add earliest/latest aggregate function for eventstats PPL command (#4212) * Add earliest/latest aggregate function for eventstats command Signed-off-by: Tomoyuki Morita <[email protected]> * update docs Signed-off-by: Tomoyuki Morita <[email protected]> * Minor refactoring Signed-off-by: Tomoyuki Morita <[email protected]> * Fix doctest Signed-off-by: Tomoyuki Morita <[email protected]> * Simplify logics Signed-off-by: Tomoyuki Morita <[email protected]> * Revert visitWindowFunction Signed-off-by: Tomoyuki Morita <[email protected]> * Add sort to some examples Signed-off-by: Tomoyuki Morita <[email protected]> * Refactor tests Signed-off-by: Tomoyuki Morita <[email protected]> * Fix argument validation error (WIP) Signed-off-by: Tomoyuki Morita <[email protected]> * Add argument validation for window functions Signed-off-by: Tomoyuki Morita <[email protected]> * Fix validation Signed-off-by: Tomoyuki Morita <[email protected]> * Fix tests Signed-off-by: Tomoyuki Morita <[email protected]> * Fix tests and refactor Signed-off-by: Tomoyuki Morita <[email protected]> * Fix test Signed-off-by: Tomoyuki Morita <[email protected]> * Fix merge issue Signed-off-by: Tomoyuki Morita <[email protected]> --------- Signed-off-by: Tomoyuki Morita <[email protected]> * Speed up aggregation pushdown for single group-by expression (#3550) * Speed up aggregation pushdown for single group-by expression Signed-off-by: Lantao Jin <[email protected]> * Add configs nullable_bucket Signed-off-by: Lantao Jin <[email protected]> * Fix IT Signed-off-by: Lantao Jin <[email protected]> * revert typo Signed-off-by: Lantao Jin <[email protected]> * Fix conflicts error Signed-off-by: Lantao Jin <[email protected]> * fix unit tests Signed-off-by: Lantao Jin <[email protected]> * Fix order Signed-off-by: Lantao Jin <[email protected]> * Fix UT Signed-off-by: Lantao Jin <[email protected]> * Fix UT in windows Signed-off-by: Lantao Jin <[email protected]> * fix compile error of conflicts Signed-off-by: Lantao Jin <[email protected]> * Add more ITs after merging push down limit to agg buckets Signed-off-by: Lantao Jin <[email protected]> * fix IT Signed-off-by: Lantao Jin <[email protected]> * address comments Signed-off-by: Lantao Jin <[email protected]> * Clear sorts in source builder for aggregation pushdown Signed-off-by: Lantao Jin <[email protected]> * Delete the TODO of v2, it's resolved now Signed-off-by: Lantao Jin <[email protected]> * fix doctest Signed-off-by: Lantao Jin <[email protected]> --------- Signed-off-by: Lantao Jin <[email protected]> * Introduce YAML formatter for better testing/debugging (#4274) * Implement YamlFormatter Signed-off-by: Tomoyuki Morita <[email protected]> * Enable YAML based plan comparison in tests Signed-off-by: Tomoyuki Morita <[email protected]> * Fix line break issue in Windows Signed-off-by: Tomoyuki Morita <[email protected]> * Minor fix in test case Signed-off-by: Tomoyuki Morita <[email protected]> * Fix line break issue Signed-off-by: Tomoyuki Morita <[email protected]> * Fix comment Signed-off-by: Tomoyuki Morita <[email protected]> --------- Signed-off-by: Tomoyuki Morita <[email protected]> * doctest: Use 1.0 branch instead of main (#4219) Signed-off-by: Simeon Widdis <[email protected]> * Fix doctest (#4292) Signed-off-by: Simeon Widdis <[email protected]> * Search Command Revamp (#4152) Signed-off-by: Vamsi Manohar <[email protected]> * `mvjoin` support in PPL Caclite (#4217) * mvjoin support in PPL Caclite Signed-off-by: ps48 <[email protected]> * fix texts Signed-off-by: ps48 <[email protected]> * update docs Signed-off-by: ps48 <[email protected]> * update doc examples Signed-off-by: ps48 <[email protected]> * rebase main, update test Signed-off-by: ps48 <[email protected]> * update test with real array fields Signed-off-by: ps48 <[email protected]> * use verifyQueryThrowsException in CalcitePPLFunctionTypeTest Signed-off-by: ps48 <[email protected]> * spotless check fix Signed-off-by: ps48 <[email protected]> * remove string,string registration for mvjoin Signed-off-by: ps48 <[email protected]> * remove string,string test Signed-off-by: ps48 <[email protected]> --------- Signed-off-by: ps48 <[email protected]> * strftime function implementation (#4106) Signed-off-by: Vamsi Manohar <[email protected]> * Add non-numeric field support for max/min functions (#4281) * add non-numeric support for max/min Signed-off-by: Ritvi Bhatt <[email protected]> * fix mixed field behavior Signed-off-by: Ritvi Bhatt <[email protected]> * update doc Signed-off-by: Ritvi Bhatt <[email protected]> * update doc Signed-off-by: Ritvi Bhatt <[email protected]> * update formatting Signed-off-by: Ritvi Bhatt <[email protected]> * add tests Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> * empty Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> * fix Signed-off-by: Ritvi Bhatt <[email protected]> * support ip type max/min Signed-off-by: Ritvi Bhatt <[email protected]> * fix formatting Signed-off-by: Ritvi Bhatt <[email protected]> * use tophitsparser Signed-off-by: Ritvi Bhatt <[email protected]> * remove v2 explain Signed-off-by: Ritvi Bhatt <[email protected]> * check for numeric fields for native max/min Signed-off-by: Ritvi Bhatt <[email protected]> * change names Signed-off-by: Ritvi Bhatt <[email protected]> * fix type checking Signed-off-by: Ritvi Bhatt <[email protected]> --------- Signed-off-by: Ritvi Bhatt <[email protected]> Signed-off-by: ritvibhatt <[email protected]> * Add `values` stats function with UDAF (#4276) * Add stats function Signed-off-by: ps48 <[email protected]> * add settings for max values Signed-off-by: ps48 <[email protected]> * update functiontypetest IT Signed-off-by: ps48 <[email protected]> * update documentation for values settings Signed-off-by: ps48 <[email protected]> * update the rst docs, remove settingsholder Signed-off-by: ps48 <[email protected]> * update AST additions Signed-off-by: ps48 <[email protected]> * updated the IT testValuesFunctionGroupBy Signed-off-by: ps48 <[email protected]> --------- Signed-off-by: ps48 <[email protected]> * Support ISO8601-formatted string in PPL (#4246) * Support parsing ISO 8601 datetime format for timestamp value Signed-off-by: Yuanchun Shen <[email protected]> * Modify tests for ISO 8601 timestamp input Signed-off-by: Yuanchun Shen <[email protected]> * Add support of iso 8601 date string to date and time - add an IT for date time comparison with iso 8601 formatted literal Signed-off-by: Yuanchun Shen <[email protected]> --------- Signed-off-by: Yuanchun Shen <[email protected]> * Push down project operator with non-identity projections into scan (#4279) * Support project push down after aggregation Signed-off-by: Heng Qian <[email protected]> * Push down project operator with non-identity projections into scan Signed-off-by: Heng Qian <[email protected]> * Fix IT Signed-off-by: Heng Qian <[email protected]> * Also changing plan from merging main Signed-off-by: Heng Qian <[email protected]> * Fix IT Signed-off-by: Heng Qian <[email protected]> * Fix 4296 Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Heng Qian <[email protected]> * Add spotless precommit hook + license check (#4306) * Add spotless precommit hook Signed-off-by: Simeon Widdis <[email protected]> * Decouple plugin spotless versions + upgrade spotless Signed-off-by: Simeon Widdis <[email protected]> * Enable license headers everywhere Signed-off-by: Simeon Widdis <[email protected]> * Remove a redundant comment Signed-off-by: Simeon Widdis <[email protected]> * Fix removed additional licenses Signed-off-by: Simeon Widdis <[email protected]> --------- Signed-off-by: Simeon Widdis <[email protected]> * Add Ryan as a maintainer (#4257) Signed-off-by: Simeon Widdis <[email protected]> * Spotless precommit: apply instead of check (#4320) * Add merge_group trigger to test workflows (#4216) * Update grammar files and developer guide (#4301) * Update grammar files and developer guide Signed-off-by: Kai Huang <[email protected]> * fix Signed-off-by: Kai Huang <[email protected]> --------- Signed-off-by: Kai Huang <[email protected]> * Fix geopoiint issue in complex data types (#4325) Signed-off-by: Vamsi Manohar <[email protected]> * [Doc] Correct the comparision table for rex doc (#4321) * [Doc] Correct the comparision table for rex doc Signed-off-by: Jialiang Liang <[email protected]> * peng - remove non support feature from comparison table Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]> * Add splunk to ppl cheat sheet (#3726) * update with latest ppl commands and function improvement Signed-off-by: Peng Huo <[email protected]> * Address comments Signed-off-by: Peng Huo <[email protected]> --------- Signed-off-by: Peng Huo <[email protected]> * Date/Time based Span aggregation should always not present null bucket (#4327) * Updating coalesce documentation (#4305) Co-authored-by: Aaron Alvarez <[email protected]> * Support serializing & deserializing UDTs when pushing down scripts (#4245) * Support serializing & deserializing UDTs Signed-off-by: Yuanchun Shen <[email protected]> * Update explain ITs Signed-off-by: Yuanchun Shen <[email protected]> * Push down UDT types as string types for comparison operators Signed-off-by: Yuanchun Shen <[email protected]> * Separate test cases and add an ignored IT Signed-off-by: Yuanchun Shen <[email protected]> * Correct the handling of UDT in CalciteScriptEngine by substituting calcite's type factory with OpenSearchTypeFactory Signed-off-by: Yuanchun Shen <[email protected]> * Fix deserialization for IP Signed-off-by: Yuanchun Shen <[email protected]> * Remove testExplainPushDownScriptsContainingUDT in v2 Signed-off-by: Yuanchun Shen <[email protected]> * Enable testLimitAfterAggregation in CalcitePPLAggregationIT Signed-off-by: Yuanchun Shen <[email protected]> * Unit test serialize map and array types Signed-off-by: Yuanchun Shen <[email protected]> * Fix deeper level deserialization of UDTs Signed-off-by: Yuanchun Shen <[email protected]> * Add a yaml test for issue 4322 Signed-off-by: Yuanchun Shen <[email protected]> * Add a test case for issue 4340 Signed-off-by: Yuanchun Shen <[email protected]> * Remove redundant classes Signed-off-by: Yuanchun Shen <[email protected]> --------- Signed-off-by: Yuanchun Shen <[email protected]> * change Anonymizer to mask PPL (#4352) * change Anonymizer Signed-off-by: xinyual <[email protected]> * fix case Signed-off-by: xinyual <[email protected]> --------- Signed-off-by: xinyual <[email protected]> * [Feature][Enhancement] Enhance patterns command with additional sample_logs output field (#4155) * Enhance patterns command with additional sample_logs output field Signed-off-by: Songkan Tang <[email protected]> * Reorder agg fields for simple_pattern Signed-off-by: Songkan Tang <[email protected]> * Test fix after previous fix to not drop group by list Signed-off-by: Songkan Tang <[email protected]> --------- Signed-off-by: Songkan Tang <[email protected]> * Optimize count aggregation performance by utilizing native doc_count in v3 (#4337) * Optimize bucket aggregation performance by utilizing native doc_count in v3 Signed-off-by: Lantao Jin <[email protected]> * fix UT Signed-off-by: Lantao Jin <[email protected]> * Fix issue of count(FIELD) Signed-off-by: Lantao Jin <[email protected]> * fix comments Signed-off-by: Lantao Jin <[email protected]> * Fix typo Signed-off-by: Lantao Jin <[email protected]> * revert the doc_count pushdown for count(FIELD) by EXPR Signed-off-by: Lantao Jin <[email protected]> * Support pushdown count aggregation in no bucket aggregation to hits.total.value Signed-off-by: Lantao Jin <[email protected]> --------- Signed-off-by: Lantao Jin <[email protected]> * No index found with given index pattern should throw IndexNotFoundException (#4369) * No index found with given index pattern should throw IndexNotFoundException Signed-off-by: Lantao Jin <[email protected]> * Add UT Signed-off-by: Lantao Jin <[email protected]> --------- Signed-off-by: Lantao Jin <[email protected]> * Push down stats with bins on time field into auto_date_histogram (#4329) * Push down stats with bins on time field into auto_date_histogram Signed-off-by: Heng Qian <[email protected]> * Prevent pushing down multiple group-by with bins in advance. Signed-off-by: Heng Qian <[email protected]> * Remove useless code Signed-off-by: Heng Qian <[email protected]> * Fix IT after merging main Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Kai Huang <[email protected]> Signed-off-by: Songkan Tang <[email protected]> Signed-off-by: Kai Huang <[email protected]> Signed-off-by: Simeon Widdis <[email protected]> Signed-off-by: Chen Dai <[email protected]> Signed-off-by: Simeon Widdis <[email protected]> Signed-off-by: Jialiang Liang <[email protected]> Signed-off-by: Ritvi Bhatt <[email protected]> Signed-off-by: ritvibhatt <[email protected]> Signed-off-by: Aaron Alvarez <[email protected]> Signed-off-by: Aaron Alvarez <[email protected]> Signed-off-by: Vamsi Manohar <[email protected]> Signed-off-by: Tomoyuki Morita <[email protected]> Signed-off-by: Lantao Jin <[email protected]> Signed-off-by: Heng Qian <[email protected]> Signed-off-by: ps48 <[email protected]> Signed-off-by: Yuanchun Shen <[email protected]> Signed-off-by: Peng Huo <[email protected]> Signed-off-by: xinyual <[email protected]> Co-authored-by: Kai Huang <[email protected]> Co-authored-by: Songkan Tang <[email protected]> Co-authored-by: Simeon Widdis <[email protected]> Co-authored-by: Chen Dai <[email protected]> Co-authored-by: Jialiang Liang <[email protected]> Co-authored-by: ritvibhatt <[email protected]> Co-authored-by: Aaron Alvarez <[email protected]> Co-authored-by: Aaron Alvarez <[email protected]> Co-authored-by: Vamsi Manohar <[email protected]> Co-authored-by: Tomoyuki MORITA <[email protected]> Co-authored-by: Lantao Jin <[email protected]> Co-authored-by: qianheng <[email protected]> Co-authored-by: Shenoy Pratik <[email protected]> Co-authored-by: Yuanchun Shen <[email protected]> Co-authored-by: Peng Huo <[email protected]> Co-authored-by: Xinyuan Lu <[email protected]>
songkant-aws
added a commit
to songkant-aws/sql
that referenced
this pull request
Sep 30, 2025
…e_logs output field (opensearch-project#4155) * Enhance patterns command with additional sample_logs output field Signed-off-by: Songkan Tang <[email protected]> * Reorder agg fields for simple_pattern Signed-off-by: Songkan Tang <[email protected]> * Test fix after previous fix to not drop group by list Signed-off-by: Songkan Tang <[email protected]> --------- Signed-off-by: Songkan Tang <[email protected]>
8 tasks
yuancu
pushed a commit
that referenced
this pull request
Sep 30, 2025
…ith additional sample_logs output field (#4155) (#4412) * [Feature][Enhancement] Enhance patterns command with additional sample_logs output field (#4155) * Enhance patterns command with additional sample_logs output field Signed-off-by: Songkan Tang <[email protected]> * Reorder agg fields for simple_pattern Signed-off-by: Songkan Tang <[email protected]> * Test fix after previous fix to not drop group by list Signed-off-by: Songkan Tang <[email protected]> --------- Signed-off-by: Songkan Tang <[email protected]> * Fix tests after merge Signed-off-by: Songkan Tang <[email protected]> * Remove unnecessary change Signed-off-by: Songkan Tang <[email protected]> * Fix unnecesary backport change from previous commit Signed-off-by: Songkan Tang <[email protected]> --------- Signed-off-by: Songkan Tang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Enhance patterns command with additional
sample_logsoutput field.patternscommand in V2 engine returnssample_logsfield in aggregation result. After Calcite is enabled, thesample_logsis not included in V3patternscommand aggregation result. This PR addresses this backward compatible change.Related Issues
Resolves #4139
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.