CHANGELOG.md

Changelog

0.12.0 (2026-07-08)

• Add unique values list for (multi)categorical columns (#250)
• Allow to analyse parquet files (#253)
• Allow to pass additional NaN values (#255)
• chore: add related test
• chore: add test value
• chore: bump urrlib3 (#251)
• chore: lint
• chore/use-uv-publish-token (#256)
• clean: fix linting errors
• feat: only run detection on unique values (#241)
• Fix datetime RFC 822 regex (#244)
• Fix unique values for too complex columns (#257)
• fix: bug on empty columns (#246)
• fix: digits
• fix: digits
• fix: fail validation if crash while loading chunks
• fix(ci): fix lint check in CI
• Rename booleen format into bool (#252)
• Update int.py
• Update siren.py
• Update siret.py
• Use iterators in any and all (#243)

0.11.2 (2026-04-03)

• feat: do not run any tests on empty columns (#240)
• feat: skip tests for mandatory labels columns (#239)

0.11.1 (2026-04-01)

• Add formats documentation and restructure docs (#237)
• Don't consider too long ints as int or float (#233)
• New format id_rnb (along with a frformat upgrade) (#234)
• Prevent NaN and inf in profile at dump time (#235)
• Prioritize utf-8 in encoding detection (#238)
• Upgrade pandas to 3.0 (#236)
• docs: update minimal python version

0.11.0 (2026-03-09)

• Add code_epci format (#214)
• Add a test to assert all formats have labels (#231)
• Allow to set custom format proportions (#213)
• Change encoding detection lib (#218)
• Fix datetime_aware.py regex pattern (#227)
• Fix datetime_rfc822.py regex pattern (#226)
• Fix examples in schema output (#224)
• Fix nb_distinct in chunked profile (#219)
• Fix nb_missing_values always returning 0 in chunked profile
• Fix scientific notation detection for negative numbers (#229)
• Fix scientific notation regex pattern (#225)
• Fix sheet name in export file name (#230)
• Fix typo in date regex pattern (#223)
• Handle empty string in percent and money detection (#222)
• Only publish when commit on main (#216)
• Remove dead code in siret.py (#228)
• Rename code_commune_insee into code_commune for better consistency (#215)
• Restrain the number of positional arguments of the main functions (#217)
• Skip step in validation if detected format is string (#232)
• Use the new kwarg name in CLI and add tests (#221)
• fix lint

0.10.4 (2026-02-09)

• Handle full NaN in short files (#210)
• Improve wording (#211)
• Upgrade minimal pandas version (#212)

0.10.3 (2026-01-27)

• Avoid considering floats as dates (#202)
• Fix casting for NaN-like values (#208)
• Fix streaming issue when analyzing from URL (#204)
• Fix validation (#205)
• Prevent NaN in headers (#209)
• Stricter column detection (#207)
• Use dependency-groups in pyproject (#203)

0.10.2 (2026-01-16)

• Dynamic versionning (#198)
• Fix tag_version.py script from template (#199)
• Handle deprecated or unknown format in validation (#200)
• Update lock (#201)

0.10.1 (2026-01-15)

• Add missing casting types (#197)
• Fix latlon labels (#195)
• Fix count in profile when chunking (#196)
• Fix hint type (#187)
• Fix references (#194)
• Improve codes ISO alpha (#192)
• Improve header score (#190)
• Improve latitude and longitude detection (#191)
• Less strict datetimes prefix (#188)
• Move stuff to Format's attributes (#189)
• Update to version 0.10.1.dev for next development cycle

0.10.0 (2025-12-04)

• Better email detection #151
• Sample can handle full NaN columns #152
• Update .gitignore #153
• Fix allowed valid proportions for geo float columns #157
• Remove Minio-related parts of the code #158
• Handle midnight in datetimes commit
• Fix datetime differentiation #162
• Bump minimum Python version from 3.9 to 3.10 #163
• New tag_version.sh script to replace Bump'x #161
• Better labels detection for some formats #168
• Restore update_version.py script and workflow #169
• Process big csv files in chunks (/!\ breaking changes) #159
• Handle column named count in profile creation #171
• Prevent NaN in headers #173
• Refactor formats structure in the repo (/!\ breaking changes) #170
• Adapt to main branch change #177
• Add lock file and unpin dependencies #166
• Use uv as the build backend instead of setuptools #165
• Allow accents and spaces in URLs #178
• Add new binary format #176
• Performances improvements #179 #180 #181 #182 #183
• float accepts scientific notation commit
• More early stops for datetimes #184

0.9.2 (2025-08-26)

• Fix profile creation #150

0.9.1 /!\ YANKED (2025-08-26)

• Remove create_example_csv_file from init to avoid noise from faker in production logs #145
• Cast columns in place and restore early detection for date(time) to cut processing time #144
• Load data with dtype=str instead of deprecated "unicode" #149
• Better sample for analysis if file is too big and therefore reduce sample size #143
• Refactor profile creation for more efficiency #145

0.9.0 (2025-07-31)

• Refactor label testing #119
• Refactor repo metadata and requirements #120 #122 #135 #136 #138
• Better URL detection #121
• For big files, analyse on sample then validate on whole file #124 #129
• Fix imports #125 #126 #127 #128
• Split aware and naive datetimes for hydra to cast them separately #130
• Validate using the testing function, to consider PROPORTIONS #131
• Remove datetime_iso format due to ambiguous cast in db (can be naive or aware) #132
• Add lonlat_wgs format and handle optional brackets for latlon_wgs #133
• Refactor format prioritizing #134
• Switch to pyproject #139
• Lint with ruff #140 #141

0.8.0 (2025-05-20)

• New function that creates a csv from a list of fields and constraints, or from a TableSchema #101
• Enable outputing loaded dataframe #102
• Better naming, hint types and minor refactors #103
• The returned dataframe has its columns properly cast to the detected types #104
• Raise an error if the encoding could not be guessed #106
• Fix CLI and minio routine #107
• Allow to only specify tests to skip ("all but...") #108
• Fix bool casting #109
• Handle csv.gz files #110
• Refactor file tests #110
• Restructure repo (breaking changes) #111
• Add validation function and associated flow #112
• Better float detection #113
• Refactor fields tests #114
• Better code waldec and add code import #116
• Better validation and refactors #117
• Fix validation #118

0.7.4 (2024-11-15)

• Enable calling main functions from base #97
• Better detection of ints and floats #94
• Better handle NaN values #96
• Reshape exemple.py, clean up code and improve changelog #98

0.7.3 (2024-10-07)

• Refactor tests import, now using folder arborescence instead of pre-made file #93
• Fix inversion (count<=>value) in profile #95

0.7.2 (2024-08-27)

• Outsource many formats to fr-format library #87
• Better date detection #89
• Update dependencies to make tests pass #81
• Update readme #81
• Hint type #81
• Minor refactors #81

0.7.1 (2024-03-27)

• Fixes after production release in hydra #80

0.7.0 (2024-03-21)

• Handle other file formats: xls, xlsx, ods (and more) and analysis through URLs #73
• Handle files with no extension (cc hydra) #79

0.6.8 (2024-01-18)

• prevent exporting NaN values in profile #72
• raise ValueError if analyzed file has various number of columns across first rows #72

0.6.7 (2024-01-15)

• Add logs for columns that would take too much time within a specific test #70
• Refactor some tests to improve performances and make detection more accurate #69
• Try alternative ways to clean text #71

0.6.6 (2023-11-24)

• Change setup.py to better convey dependencies #67

0.6.5 (2023-11-17)

• Change encoding detection for faust-cchardet (forked from cchardet) #66

0.6.4 (2023-10-18)

• Better handling of ints and floats (now not accepting blanks and "+" in string) #62

0.6.3 (2023-03-23)

• Faster routine #59

0.6.2 (2023-02-10)

• Catch OverflowError for latitude and longitude checks #58

0.6.0 (2023-02-10)

• Add CI and upgrade dependencies #49
• Shuffle data before analysis #56
• Better discrimination between code_departement and code_region #56
• Add schema in output analysis #57

0.4.7 #51

• Allow possibility to analyze entire file instead of a limited number of rows #48
• Better boolean detection #42
• Differentiate python types and format for date and datetime #43
• Better code_departement and code_commune_insee detection #44
• Fix header line (header_row_idx) detection #44
• Allow possibility to get profile of csv #46

0.4.6 #39

• Fix tests
• Prioritise lat / lon FR detection over more generic lat / lon.
• To reduce false positives, prevent detection of the following if label detection is missing: ['code_departement', 'code_commune_insee', 'code_postal', 'latitude_wgs', 'longitude_wgs', 'latitude_wgs_fr_metropole', 'longitude_wgs_fr_metropole', 'latitude_l93', 'longitude_l93']
• Lower threshold of label detection so that if one relevant is detected in the label, it boosts the detection score.
• Add ISO country alpha-3 and numeric detection
• include camel case parsing in _process_text function
• Support optional brackets in latlon format

0.4.5 #29

• Use netloc instead of url in location dict

0.4.4 [#24] (#28)

• Prevent crash on empty CSVs
• Add optional arguments encoding and sep to routine and routine_minio functions
• Field detection improvements (code_csp_insee and datetime RFC 822)
• Schema generation improvements with examples

0.4.3 [#24] (#24)

• Add uuid and MongoID detection
• Add new function dedicated to interaction with minio data
• Add table schema automatic generation (only on minio data)
• Modification of calculated score (consider label detection as a boost for score)

0.4.2 [#22] (#22)

Add type detection by header name

0.4.1 [#19] (#19)

Fix bug

• num_rows was causing problem when it was fix to other value than default - Fixed

0.4.0 [#18] (#18)

Add detailed output possibility

Details :

• two modes now for output report : "LIMITED" and "ALL"
• "ALL" option give user information on found proportion for each column types and each columns

0.3.0 [#15] (#15)

Fix bugs

Details :

• Facilitate ML Integration
• Add column types detection
• Fix documentation

0.2.1 - #2

Add continuous integration

Details :

• Add configuration for CircleCI
• Add CONTRIBUTING.md
• Push automatically new versions to PyPI
• Use semantic versioning

0.2 - #1

Port from python2 to python3

Details :

• Add license AGPLv3
• Update requirements

0.1