DBFxSQL

Configure once, sync automatically!

 

✨ Overview

DBFxSQL enables seamless data consistency between legacy DBF (dBase) files and modern SQL databases during migration projects. Instead of managing two separate data systems manually, DBFxSQL automatically synchronizes changes in both directions, eliminating data reconciliation headaches.

Suitable for:

• Organizations migrating from DBF to SQL while maintaining production systems.
• Development teams building new SQL systems alongside existing DBF infrastructure.
• One-time clean migrations without ongoing DBF dependencies.

 

🔌 Installation

1. Check your Python version, we use python = "^3.12":

python --version

2. Clone the repository:

git clone https://github.com/joelabreurojas/DBFxSQL.git

3. Create and activate a virtual environment.

4. Install the source code:

pip install DBFxSQL/

5. Run the tool:

dbfxsql

 

💻 Usage

1. Create a DBF file:

dbfxsql create -s users.dbf -f id "N(20,0)" -f name "C(20)"

2. Insert data into DBF:

dbfxsql insert -s users.dbf -f id 1 -f name "John Doe"

3. Create SQL database and table:

dbfxsql create -s company.sql -t users -f id 'integer primary key' -f name text

4. One-time migration:

dbfxsql migrate --notify

5. Continuous synchronization:

dbfxsql sync --notify

Tip

Use the --help flag to see all available commands and options.

Example: dbfxsql <command> --help

 

⚙️ Configuration

DBFxSQL uses TOML configuration files to define:

• Database engines and their file extensions.
• Folder paths to monitor.
• Table relationships and synchronization priorities.
• Field mappings between DBF and SQL schemas.

The tool automatically creates a default configuration on the first run at ~/.config/DBFxSQL/config.toml.

 

📌 Disclaimer

Before using, create backups of your DBF files and SQL databases.

Known limitations:

• Requires local file system access.
• MSSQL connection issues in database manipulation commands.
• Some advanced features are still under development.

Use at your own risk and carefully verify data integrity after operations.

 

📝 To do

Required:

• Fix connection in DB manipulation commands for MSSQL.

Desirable:

• Add a changelog.
• Add loading bar during migration.
• Add option to initialize triggers/procedures in cli.
• Add option to listen alternative files in config.
• Add option to specify migration/sync order in config.
• Add errors for wrong config.
• Add FIELDS option to filter read requests.
• Add option to specify field target for indirect table relations in config.
• Balance the number of rows between indirect table relationships (large over small).
• Add CDC into SQL layers.
• Add Command Query Responsibility Segregation (CQRS) pattern.
• Refactor store procedure write_file (use CLR procedures).
• Refactor read queries by group origin tables by destiny tables for migration/sync optimizations.
• Validate existence of received field type.
• Validate KeyErrors for invalid fields.
• Validate type lengths and names for consistency between DBF and SQL.
• Validate existence of target fields during sync.
• Release as a Python library.

 

👐 Contribute

Improvements?

• Don't hesitate to create a PR.

Problems?

• Feel free to open a new issue!

 

❤️ Gratitude

Special thanks to the following projects for making this tool possible:

• Flask Boilerplate - Python Structure Guide by Ezequiel L. Castaño.
• DBF library - Pure Python DBF reader/writer by Ethan Furman.
• Watchfiles library - Simple, modern and fast file watching and code reload in Python by Samuel Colvin.
• Click library - A Python command line interface toolkit by Pallets Organization.