Code

Avro codec enhancements#6965

Code

Open

jecsand838 wants to merge 45 commits intoapache:mainapache/arrow-rs:mainfrom elastiflow:avro-codecelastiflow/arrow-rs:avro-codecCopy head branch name to clipboard

+2,188-384

Commits

Commits on Dec 28, 2024

• Added basic support for arrow -> avro codec along with beginnings of avro writer.

  jecsand838

  committed
  7d4e6fd

  Copy full SHA for 7d4e6fd

Commits on Dec 30, 2024

• Added codec support + tests for:

  Show description for 36d56a9
  jecsand838

  committed
  36d56a9

  Copy full SHA for 36d56a9

Commits on Dec 31, 2024

• Added reader record decoder support for non-null Enum, Map, and Decimal types.

  Show description for 36b4b73
  jecsand838

  committed
  36b4b73

  Copy full SHA for 36b4b73

• Added reader record decoder support for non-null Enum, Map, and Decimal types.

  Show description for 9d0bf4c
  jecsand838

  committed
  9d0bf4c

  Copy full SHA for 9d0bf4c

• Added reader record decoder support for non-null Enum, Map, and Decimal types.

  Show description for 082581a
  jecsand838

  committed
  082581a

  Copy full SHA for 082581a

• Added null support

  Show description for 6647b25
  jecsand838

  committed
  6647b25

  Copy full SHA for 6647b25

• * Reader decoder Support for nullable types.

  Show description for 9cfda09
  jecsand838

  committed
  9cfda09

  Copy full SHA for 9cfda09

• * Minor Cleanup

  Show description for 84ffb62
  jecsand838

  committed
  84ffb62

  Copy full SHA for 84ffb62

• * Added record decoder support for the following types:

  Show description for 8600680
  jecsand838

  committed
  8600680

  Copy full SHA for 8600680

Commits on Jan 3, 2025

• Merge pull request #1 from elastiflow/avro-codec-improvements

  Show description for 2792037
  svencowart

  authored
  2792037

  Copy full SHA for 2792037

• Merge branch 'apache:main' into avro-codec

  jecsand838

  authored
  3029efc

  Copy full SHA for 3029efc

Commits on Jan 6, 2025

• Merge branch 'apache:main' into avro-codec

  jecsand838

  authored
  01e5224

  Copy full SHA for 01e5224

• Minor cleanup

  Show description for 8331058
  jecsand838

  committed
  8331058

  Copy full SHA for 8331058

Commits on Jan 7, 2025

• chore: import cleanup and formatting

  svencowart

  committed
  c54de48

  Copy full SHA for c54de48

• chore: simplifies and cleans code

  svencowart

  committed
  fc696c8

  Copy full SHA for fc696c8

Commits on Jan 10, 2025

• ran linter

  Show description for 81d7bba
  jecsand838

  committed
  81d7bba

  Copy full SHA for 81d7bba

• Merge branch 'apache:main' into avro-codec

  jecsand838

  authored
  03bd5f0

  Copy full SHA for 03bd5f0

Commits on Jan 13, 2025

• Merge branch 'apache:main' into avro-codec

  jecsand838

  authored
  eeabc49

  Copy full SHA for eeabc49

• Removed Avro writer module and Avro writer related logic from codec.rs

  Show description for 1bc9a51
  jecsand838

  committed
  1bc9a51

  Copy full SHA for 1bc9a51

Commits on Jan 14, 2025

• chore: clean up UUIDs and Durations

  svencowart

  committed
  b157584

  Copy full SHA for b157584

• chore: clean up UUIDs Durations

  svencowart

  committed
  8d0fe77

  Copy full SHA for 8d0fe77

• add aliases to record fields

  svencowart

  committed
  0496dcf

  Copy full SHA for 0496dcf

• Merge branch 'apache:main' into avro-codec

  jecsand838

  authored
  7208f78

  Copy full SHA for 7208f78

Commits on Jan 15, 2025

• * Fixed size issue in codec.rs fixed type decimal

  Show description for 92e105f
  jecsand838

  committed
  92e105f

  Copy full SHA for 92e105f

Commits on Jan 17, 2025

• * Removed Avro writer specific maybe_add_namespace function from codec.rs

  Show description for 51586d8
  jecsand838

  committed
  51586d8

  Copy full SHA for 51586d8

• Aligns mapping between avro and codec and tests all paths of arrow_field_to_avro_field

  svencowart

  committed
  a51b202

  Copy full SHA for a51b202

Commits on Jan 18, 2025

• * Organized Types to match Avro spec ordering

  Show description for 5585a87
  jecsand838

  committed
  5585a87

  Copy full SHA for 5585a87

• * Fixed failing test_enum_decoding test due to constructing Codec::Enum with an empty list of symbols.

  Show description for 26faf5b
  jecsand838

  committed
  26faf5b

  Copy full SHA for 26faf5b

• * Optimized avro codec.rs data_type Self::Decimal case:

  Show description for cf45fcf
  jecsand838

  committed
  cf45fcf

  Copy full SHA for cf45fcf

Commits on Jan 29, 2025

• Fixed hyperlink linter errors

  Show description for 75e0ad1
  jecsand838

  committed
  75e0ad1

  Copy full SHA for 75e0ad1

Commits on Feb 2, 2025

• Implemented improved defaults

  Show description for f93331d
  jecsand838

  committed
  f93331d

  Copy full SHA for f93331d

Commits on Feb 3, 2025

• Implemented test_nonnullable_impala in arrow-avro/src/reader/mod.rs

  Show description for eee3948
  jecsand838

  committed
  eee3948

  Copy full SHA for eee3948

Commits on Feb 6, 2025

• Implemented test_nullable_impala testcase in arrow-avro/src/reader/mod.rs

  Show description for 6d1ae49
  jecsand838

  committed
  6d1ae49

  Copy full SHA for 6d1ae49

Commits on Feb 7, 2025

• linter

  Show description for 332b3fb
  jecsand838

  committed
  332b3fb

  Copy full SHA for 332b3fb

• Added a strict_model parameter to the Avro read_file method for enforcement strictness of avro schema, i.e. Impala usecase.

  Show description for fadbf5e
  jecsand838

  committed
  fadbf5e

  Copy full SHA for fadbf5e

Commits on Feb 8, 2025

• Added test_nulls_snappy and `test_repeated_no_annotation to arrow-avro/src/reader/mod.rs

  Show description for dbe8555
  jecsand838

  committed
  dbe8555

  Copy full SHA for dbe8555

• Added remaining Avro read_file tests to arrow-avro/src/reader/mod.rs

  Show description for ec6d0ca
  jecsand838

  committed
  ec6d0ca

  Copy full SHA for ec6d0ca

Commits on Feb 9, 2025

• Added Avro Decoder, ReaderBuilder, and Reader

  Show description for dbdf79a
  jecsand838

  committed
  dbdf79a

  Copy full SHA for dbdf79a

Commits on Feb 11, 2025

• Implemented Avro Decoder, ReaderBuilder, and Reader + minor code cleanup

  Show description for fffcbed
  jecsand838

  committed
  fffcbed

  Copy full SHA for fffcbed

• Merge branch 'main' into avro-codec

  jecsand838

  authored
  e5c9dd6

  Copy full SHA for e5c9dd6

Commits on Feb 12, 2025

• Optimized the code in Avro codec.rs and record.rs files.

  Show description for 3dfe0df
  jecsand838

  committed
  3dfe0df

  Copy full SHA for 3dfe0df

• Small arrow-avro lib.rs update

  Show description for 0de2eda
  jecsand838

  committed
  0de2eda

  Copy full SHA for 0de2eda

Commits on Feb 15, 2025

• Merge branch 'main' into avro-codec

  jecsand838

  authored
  0bea7cc

  Copy full SHA for 0bea7cc

• Reverted all changes not related to enhanced Avro codec and record decoder support + .avro file tests

  Show description for ad34fb4
  jecsand838

  committed
  ad34fb4

  Copy full SHA for ad34fb4

Commits on Feb 16, 2025

• Fixed dev-dependency lint issue and removed some currently unused functions from codec.rs

  jecsand838

  committed
  3424300

  Copy full SHA for 3424300