Add database export and import documentation (#940)

## Goal
Introduce database export and import documentation with an extended
migration guide between TypeDB 2.x and TypeDB 3.x.

## Implementation
Fill the database export / import page with theory and usage details. 

Redesign the 2.x to 3.x migration manual to have multiple pages:
* index for the introduction;
* a separate page for differences between 2.x and 3.x;
* a separate page with the migration guide and a detailed example to
work with different errors occurring along the way.

Author: farost

manual/modules/ROOT/pages/maintenance/export_import.adoc

@@ -0,0 +1,100 @@
+= Export and import
+:page-aliases: {page-version}@manual::configuring/export.adoc, {page-version}@manual::configuring/import.adoc, {page-version}@manual::configuring/import_export.adoc
+
+In this guide, you'll see how to export and import the schema and data of a TypeDB database.
+
+[IMPORTANT]
+====
+This feature is introduced in TypeDB 3.4.0. If you want to export or import a database from earlier versions of TypeDB 3.x, simply update your server and client and follow this manual.
+
+You can import your databases exported from TypeDB 2.x following xref:{page-version}@manual::migration/2_to_3/process.adoc[this guide].
+====
+
+== Understanding export and import
+
+TypeDB database can be represented as two files:
+
+[#_files]
+1. A text file with its TypeQL schema description: a complete xref:{page-version}@manual::schema/schema_structure.adoc[`define` query] for the whole schema.
+2. A binary file with its data.
+
+An exported database can be imported into another version of TypeDB using these two files.
+This is the best tool you can use to migrate a database between two incompatible versions of TypeDB (e.g., a migration xref:{page-version}@manual::migration/2_to_3/process.adoc[from TypeDB 2.x to TypeDB 3.x]).
+
+Both operations are performed through the network, but the server and the client can be used on the same host.
+
+Examples below use xref:{page-version}@manual::tools/console.adoc[TypeDB Console] as the most accessible client for these operations.
+Please check the xref:{page-version}@drivers::index.adoc[Drivers API references] to explore available programmatic commands (e.g., xref:{page-version}@drivers::rust/api-reference.adoc[Rust]).
+
+[#_export]
+== Export
+
+Database export allows a client to download xref:#_files[database schema and data files] from a TypeDB server for future import to the same or higher TypeDB version.
+
+The files are created on the client side.
+The exported binary file might be up to two times larger than the original database.
+If the client connects to a remote host, the server's disk remains unchanged, thus introducing zero risks to the stability of the server.
+Please ensure the client's disk has enough free capacity to store a file of this size.
+
+While the database data is being exported, parallel queries are allowed, but none of them will affect the exported data thanks to TypeDB's xref:{page-version}@manual::queries/transactions.adoc[transactionality].
+However, the database will not be available for such operations as deletion.
+
+[NOTE]
+====
+The files can be incomplete and corrupted if the export operation is interrupted by user.
+====
+
+=== Perform through TypeDB Console
+
+1. Make sure that your TypeDB Server is running.
+
+2. Follow xref:{page-version}@manual::tools/console.adoc[this guide] for TypeDB Console set up and connection.
+
+3. Execute `database export database-name schema.typeql data.typedb`, where
+
+- `database-name` is the name of the exported database;
+- `schema.typeql` is the exported schema definition file path;
+- `data.typedb` is the exported data file path.
+
+4. The result (either `Successfully exported database` or an error) will be printed to the Console.
+
+== Import
+
+Database import allows a client to upload previously exported xref:#_files[database schema and data files] to a TypeDB server.
+
+For your safety, the imported database should not exist on the target server.
+It is possible to assign any new name to the imported database.
+
+[NOTE]
+====
+The database will be incomplete (and thus not recommended for usage) if the import operation is interrupted.
+It is expected to be automatically removed from the disk, but a small number of unexpected technical issues can prevent this from happening.
+A server restart will resolve any related issues.
+====
+
+=== Perform through TypeDB Console
+
+1. Make sure that your TypeDB Server is running.
+
+2. Follow xref:{page-version}@manual::tools/console.adoc[this guide] for TypeDB Console set up and connection.
+
+3. Execute `database import database-name schema.typeql data.typedb`, where
+
+- `database-name` is the new name of the imported database;
+- `schema.typeql` is the imported schema definition file path;
+- `data.typedb` is the imported data file path.
+
+4. The result (either `Successfully imported database` or an error) will be printed to the Console.
+
+== Troubleshooting
+
+Follow this checklist if any errors occur:
+
+* Ensure that you use xref:{page-version}@manual::tools/console.adoc#version-compatiblity[compatible] TypeDB Server and TypeDB Console.
+* Ensure that the correct file paths were specified for the operations.
+* Ensure that the schema has been imported correctly to the new database.
+Use xref:{page-version}@manual::studio.adoc[TypeDB Studio] or xref:{page-version}@manual::console.adoc[TypeDB Console] to check the schema after import.
+* If you import a TypeDB 2.x database, some errors may be expected.
+Check out xref:{page-version}@manual::migration/2_to_3/process.adoc[this page] for a detailed guide.
+
+If any errors persist, please join one of our communities and ask for assistance.

manual/modules/ROOT/pages/maintenance/exports.adoc

@@ -1,38 +1,17 @@
-= TypeDB 2.x to 3.x
-
-TypeDB 3.x is a major leap forward, offering significantly improved performance and introducing a wide range of changes across almost every aspect of the database.
-This page outlines the key updates you should know, alongside recommendations for server and driver upgrades.
-
-== Migration process
-
-1. **Review the <<_temporarily_missing, missing features>> list** before starting your migration.
-Some features might not yet be available in 3.0, and you may need to wait for a newer version to proceed.
-1. **Understand the scope of changes** by referring to the <<_major_changes, major changes>> section.
-1. **Export your schema and data** from TypeDB 2.x using the xref:2.x@manual::configuring/export.adoc[export manual].
-1. **Data migration guide and tools are in progress.** Full migration of a production database is currently unsupported. **Stay tuned for updates!**
-1. *(Optional)* Refresh your knowledge of TypeDB and TypeQL by following the updated xref:{page-version}@home::quickstart.adoc[Quickstart Guide].
-1. **Set up TypeDB 3.0** by following the xref:{page-version}@manual::configure/index.adoc[Configuration Manual].
-The setup process remains largely unchanged from 2.x.
-1. **Manually update your schema definition file.** Use error messages during application to guide adjustments.
-The <<_major_changes, major changes>> section highlights the most common updates needed.
-- Rewrite rules in your schema as functions.
-Refer to the xref:{page-version}@typeql::functions/index.adoc[Functions Documentation] and xref:{page-version}@manual::schema/functions.adoc[Manual] for guidance on the new abstraction model.
-1. *(Optional)* Update your supporting queries using the xref:{page-version}@manual::schema/index.adoc[Schema] and xref:{page-version}@manual::CRUD/index.adoc[CRUD] manuals.
-1. *(Optional)* Update your drivers.
-Consult the xref:{page-version}@drivers::index.adoc[Drivers Manual] and the <<_major_changes_drivers, major driver changes>> list for details.
-
-[#_major_changes]
-== Major changes
+= TypeDB 2.x to TypeDB 3.x: what's changed
+
+This page outlines the key updates of TypeDB 3.x compared to 2.x you should be aware of while transitioning between these versions.
 
 [NOTE]
 ====
-This section highlights **changes from TypeDB 2.x**, not all new features in 3.0. Explore the xref:{page-version}@manual::index.adoc[Manual], xref:{page-version}@typeql::index.adoc[TypeQL Documentation], and https://github.com/typedb/typedb/releases[release notes] for the full picture.
+This section highlights **changes from TypeDB 2.x**, not all new features in 3.x.
+Explore the xref:{page-version}@manual::index.adoc[Manual], xref:{page-version}@typeql::index.adoc[TypeQL Documentation], and https://github.com/typedb/typedb/releases[release notes] for the full picture.
 Stay tuned for upcoming blogs showcasing new features!
 ====
 
 Click on a feature below for detailed documentation and examples.
 
-=== TypeDB and TypeQL
+== TypeDB and TypeQL
 
 * **Sessions Removed:** Connections to TypeDB now only require opening a `schema`, `write`, or `read` xref:{page-version}@manual::queries/transactions.adoc[transaction].
 * **Integer Value Type Renamed:** `long` is now xref:{page-version}@typeql::values/integer.adoc[`integer`].
@@ -61,9 +40,10 @@ Explicitly define cardinalities for traits expecting multiple instances to avoid
 * **Rules and Inference Replaced:** Rules are now replaced by xref:{page-version}@typeql::functions/index.adoc[functions], requiring explicit functions execution instead of inference, but offering more abstraction and flexibility.
 
 [#_major_changes_drivers]
-=== TypeDB Drivers
+== TypeDB Drivers
 
 **API Simplification** for smoother UX and better maintenance.
+Visit the xref:{page-version}@drivers::index.adoc[Drivers manual] to access their full documentation.
 
 - Now, all schema and data fetching and modifications are performed through a single `transaction.query("your query")` interface.
 - No more `query().insert()` and `query().define()`.
@@ -80,21 +60,15 @@ Take into account that some of your processes can be blocked if you previously r
 
 === TypeDB and TypeQL
 
-* Database import and export tools.
-* Configuration files (use CLI as a temporary solution).
 * Instantiation restriction for inherited `owns` and `plays` (used to `as`).
-A better way is planned to be released in the future (NOTE: you don't need the `as` keyword to <<#_specialize, specialize>> an ownership of an inherited super attribute type with an ownership of its subtype).
+An enhanced approach is in development (NOTE: you don't need the `as` keyword to <<#_specialize, specialize>> an ownership of an inherited super attribute type with an ownership of its subtype, but you cannot "hide" the original definition from inheritance).
 * Scaling support for Cloud and Enterprise editions.
 
 === TypeDB Drivers
 
 * Node.js, C, C++, and C# drivers are not yet available.
 Consider using one of xref:{page-version}@drivers::index.adoc[the available languages] or xref:{page-version}@drivers::http/index.adoc[the HTTP Endpoint].
 
-=== TypeDB Studio
-
-* **Graph visualizer** and **Type browser** are temporarily unavailable, but will be reimplemented and enhanced in the new TypeDB Studio soon.
-
 [#_having_troubles]
 == Having troubles?
 

manual/modules/ROOT/pages/maintenance/import_export.adoc

@@ -0,0 +1,20 @@
+= TypeDB 2.x to TypeDB 3.x
+
+TypeDB 3.x is a major leap forward, offering significantly improved performance and introducing a wide range of changes across almost every aspect of the database.
+
+These pages outline the key changes and upgrade instructions to make your migration from an older version of TypeDB as smooth as possible.
+
+[cols-2]
+--
+.xref:{page-version}@manual::migration/2_to_3/diff.adoc[]
+[.clickable]
+****
+Explore all main changes between the two major versions of TypeDB.
+****
+
+.xref:{page-version}@manual::migration/2_to_3/process.adoc[]
+[.clickable]
+****
+Upgrade TypeDB 2.x to TypeDB 3.x following a step-by-step guide.
+****
+--

manual/modules/ROOT/pages/migration/2_to_3.adoc renamed to manual/modules/ROOT/pages/migration/2_to_3/diff.adoc

@@ -0,0 +1,177 @@
+= TypeDB 2.x to TypeDB 3.x: migration process
+
+This page gives recommendations for server and driver upgrades between the two major versions of TypeDB, as well as a step-by-step example of such a migration.
+
+== Migration plan
+
+1. **Understand the scope of changes** by referring to the xref:{page-version}@manual::migration/2_to_3/diff.adoc[major changes] page.
+1. **Review the xref:{page-version}@manual::migration/2_to_3/diff.adoc#_temporarily_missing[missing features] list** before starting your migration.
+Some features might not yet be available in 3.x, and you may need to wait for a newer version to proceed.
+1. *(Optional)* Refresh your knowledge and adjust to the new versions of TypeDB and TypeQL by following the updated xref:{page-version}@home::quickstart.adoc[Quickstart Guide].
+1. **Set up TypeDB 3.x** by following the xref:{page-version}@manual::configure/index.adoc[Configuration Manual].
+1. **Migrate your TypeDB 2.x database** following the xref:#_step_by_step[step-by-step instructions below].
+The setup process remains largely unchanged from 2.x.
+1. *(Optional)* Update your supporting queries using the xref:{page-version}@manual::schema/index.adoc[Schema] and xref:{page-version}@manual::CRUD/index.adoc[CRUD] manuals.
+1. *(Optional)* Update your drivers.
+Consult the xref:{page-version}@drivers::index.adoc[Drivers Manual] and the xref:{page-version}@manual::migration/2_to_3/diff.adoc#_major_changes_drivers[major driver changes] list for details.
+
+[#_step_by_step]
+== Step-by-step example
+
+This section showcases a step-by-step example of migrating a database between TypeDB 2.x and TypeDB 3.x using xref:{page-version}@manual::maintenance/export_import.adoc[major changes][export and import].
+
+This example uses a database called `social-network`.
+It contains an arbitrary set of data and the 2.x schema provided below.
+
+[#_schema_reference]
+.social-network's 2.x schema
+[%collapsible]
+====
+.social-network.typeql
+[,typeql]
+----
+include::{page-version}@manual:resources:partial$migration/2_to_3/schema-2x.tql[tag=full-query]
+----
+====
+
+[#_prepare_data]
+=== Optional: resolve data conflicts
+
+While most of the changes in TypeDB 3.x affect only the schema, there is also one very important change in data, which can lead to errors on the <<_import, import stage>> of this guide.
+
+**Attributes can no longer own attributes** in TypeDB 3.x.
+If your data contains attributes owning attributes, you will see an error like:
+
+[source]
+----
+Invalid migration item received: attributes cannot own attributes in this version of TypeDB (deprecated).
+----
+
+In this situation, an additional step is required before proceeding, and it will require modifications of the original 2.x's database.
+
+==== Move attributed owned by attributes
+
+This is an example of migration of attributes owned by attributes for the <<_schema_reference, referenced schema>>.
+
+Our goal is to move `bio-version` from `bio` to `page` owning the `bio` attributes.
+This way, both `bio` and `bio-version` will be attached to the same entity (note that it's only one of the ways to redesign this schema).
+
+First, using a schema write transaction, define the additional required trait of `page`:
+
+[,typeql]
+----
+include::{page-version}@manual:resources:partial$migration/2_to_3/attribute-migration-queries.tql[tag=step1-define-entity-ownership]
+----
+
+Then, using a data write transaction, perform the data moving between `bio`s and `page`s:
+
+[,typeql]
+----
+include::{page-version}@manual:resources:partial$migration/2_to_3/attribute-migration-queries.tql[tag=step2-move-data]
+----
+
+Finally, verify that the data is moved correctly.
+As an additional step of verification, you can try undefining the original ownership to verify that it's now redundant:
+
+[,typeql]
+----
+include::{page-version}@manual:resources:partial$migration/2_to_3/attribute-migration-queries.tql[tag=step3-undefine-attribute-ownership]
+----
+
+Now your data is ready to get migrated to TypeDB 3.x!
+
+[#_export]
+=== Export the database from TypeDB 2.x
+
+Use xref:2.x@manual::configuring/export.adoc#_export[2.x export] to export your TypeDB 2.x database as a schema definition file and a data file.
+
+While your server is running, execute `typedb server export` using the same server binary:
+
+[source,console]
+----
+typedb server export --database=social-network --port=1729 --schema=social-network.typeql --data=social-network.typedb
+----
+
+[#_prepare_schema]
+=== Prepare the new schema
+
+**Manually update your schema definition file.**
+
+- The xref:{page-version}@manual::migration/2_to_3/diff.adoc[major changes] page highlights the most common updates needed.
+- For simplicity, rules can be completely removed.
+Additional queries can be run to define functions as their replacements later.
+
+This step can require a couple of iterations until you reach the final state of your new schema you will be satisfied with.
+Use error messages during application to guide adjustments.
+
+This is a sample example of the updated ``social-network``'s schema, written in TypeQL 3.x:
+
+.social-network's 3.x schema
+[%collapsible]
+====
+.social-network-3x.typeql
+[,typeql]
+----
+include::{page-version}@manual:resources:partial$migration/2_to_3/schema-3x.tql[tag=full-query]
+----
+
+Noticeable highlights:
+
+- Attribute types can be subtyped even if they are not abstract.
+- Attributes can no longer own attributes.
+See `bio-version` for an example of such redesign.
+- `long` is now `integer`.
+- We can restrict cardinalities of `relates`, `owns`, and `plays` using the `@card` annotation.
+Default cardinality values protect your data, but they may not suite your needs. Consider overriding them explicitly.
+- We can redeclare traits like `owns` on subtypes to put additional constraints on them.
+- While there are no `as` keyword for `owns` and `plays`, in most cases, like ours, it's not needed.
+E.g., the previously overridden `owns id` does not require overriding in 3.x as it's an ownership of an abstract attribute, so it cannot be instantiated without type specialization (e.g., `owns post-id`).
+- More constraints can be moved from your application logic to the schema: see the implication of `@values`, an enum-like constraint of TypeQL.
+
+If you want to put data constraints, but your data does not yet satisfy them, you can start with a relaxed schema and then add the constraints required via additional `define` queries.
+====
+
+[#_import]
+=== Import the database into TypeDB 3.x
+
+Launch a xref:{page-version}@manual::install/index.adoc[TypeDB 3.x server] and connect to it using a compatible xref:{page-version}@manual::tools/console.adoc[TypeDB Console].
+
+In TypeDB Console, execute a `database import` command:
+
+[source,console]
+----
+database import social-network /path-to-updated-schema/social-network-3x.typeql /path-to-2.x-export/social-network.typedb
+----
+
+[NOTE]
+====
+You can operate other server's databases without restrictions while the new database is being imported.
+====
+
+In case of successful migration, you will see:
+
+[source]
+----
+Successfully imported database.
+----
+
+Otherwise, a descriptive error will be shown.
+After fixing this error, retry the same command.
+
+[#_after_import]
+=== After import
+
+From this point, your database is completely functional.
+Its schema and data are synchronized and technically correct.
+
+We recommend running a few read queries to verify that the migration was logically successful and familiarize yourself with the updated TypeQL and your new schema.
+
+If necessary, define TypeQL functions and update any queries that previously relied on the removed rules to use the new functions instead.
+
+Visit xref:{page-version}@manual::migration/2_to_3/diff.adoc[] and xref:{page-version}@manual::index.adoc[] to learn more about TypeDB 3.x.
+Enjoy your journey!
+
+[#_having_troubles]
+== Having troubles?
+
+For troubleshooting tips and guidance, refer to the xref:{page-version}@manual::troubleshooting/index.adoc[Troubleshooting Manual].