Skip to content

Commit 926d4c1

Browse files
Sameer6305KaifAhmad1
Sameer6305
and
KaifAhmad1
authored
docs: add choose-your-module onboarding guide (#651)
* docs: add choose-your-module onboarding guide * fix(docs): correct export code examples against actual API signatures - export_to_rdf() returns a string; use export() for file output - format="json-ld" is invalid; correct value is "jsonld" - ParquetExporter/LPGExporter/ArangoAQLExporter take file_path as a required positional arg, not output= / output_dir= kwargs - ArangoAQLExporter().export(graph) was missing file_path entirely, which would raise TypeError at runtime - Remove misleading 'with provenance embedded' comment (no such param) --------- Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
1 parent 12d61b9 commit 926d4c1

4 files changed

Lines changed: 339 additions & 19 deletions

File tree

‎docs/choose-your-module.md‎

Lines changed: 327 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,327 @@
1+
---
2+
title: "Choose the Right Module"
3+
description: "Map your goal to the right Semantica module in under 30 seconds."
4+
icon: "compass"
5+
---
6+
7+
<Info>
8+
Every module works independently — import only what you need. This page maps developer goals to starting points. The [Module Reference](modules) covers every module in depth.
9+
</Info>
10+
11+
## Quick Reference
12+
13+
Find your goal below. The **Module** column is your import path; **Key class** is what you instantiate first.
14+
15+
| I want to... | Module | Key class |
16+
| :------------ | :------ | :--------- |
17+
| Load a PDF, DOCX, HTML, CSV, or archive | `ingest` | `FileIngestor` |
18+
| Crawl a website | `ingest` | `WebIngestor` |
19+
| Load Parquet files or partitioned datasets | `ingest` | `ParquetIngestor` |
20+
| Ingest XML with schema validation | `ingest` | `XMLIngestor` |
21+
| Ingest from SQL, Snowflake, Kafka, or email | `ingest` | `DBIngestor`, `SnowflakeIngestor`, `StreamIngestor` |
22+
| Extract clean text and tables from a document | `parse` | `DocumentParser` |
23+
| Parse complex PDFs with OCR or multi-column layout | `parse` | `DoclingParser` |
24+
| Chunk text for embedding or RAG | `split` | `TextSplitter` |
25+
| Normalize text, dates, entities, or encodings | `normalize` | `TextNormalizer`, `EntityNormalizer` |
26+
| Find named entities (people, orgs, locations) in text | `semantic_extract` | `NERExtractor` |
27+
| Extract typed relationships from text | `semantic_extract` | `RelationExtractor` |
28+
| Extract RDF subject–predicate–object triplets | `semantic_extract` | `TripletExtractor` |
29+
| Build a queryable knowledge graph | `kg` | `GraphBuilder` |
30+
| Add time-validity (`valid_from` / `valid_until`) to facts | `kg` | `TemporalGraphQuery` |
31+
| Run graph algorithms (centrality, communities, paths) | `kg` | `GraphAnalyzer`, `CentralityCalculator` |
32+
| Generate vector embeddings | `embeddings` | `EmbeddingGenerator` |
33+
| Store and search vectors | `vector_store` | `VectorStore` |
34+
| Persist a graph in Neo4j or FalkorDB | `graph_store` | `Neo4jStore`, `FalkorDBStore` |
35+
| Store RDF triples and query with SPARQL | `triplet_store` | `TripletStore` |
36+
| Deduplicate entities across sources | `deduplication` | `DuplicateDetector`, `EntityMerger` |
37+
| Detect and resolve contradictory facts | `conflicts` | `ConflictDetector`, `ConflictResolver` |
38+
| Give an AI agent persistent memory | `context` | `AgentContext` |
39+
| Ground LLM responses in a knowledge graph (GraphRAG) | `context` | `AgentContext.query_with_reasoning()` |
40+
| Record AI decisions with a full audit trail | `context` | `AgentContext.record_decision()` |
41+
| Search past decisions before making a new one | `context` | `AgentContext.find_precedents()` |
42+
| Trace the causal chain of a decision | `context` | `AgentContext.get_causal_chain()` |
43+
| Track where every fact came from (W3C PROV-O) | `provenance` | `ProvenanceManager` |
44+
| Version a graph with checksums and rollback | `change_management` | `TemporalVersionManager` |
45+
| Auto-generate an OWL schema from a graph | `ontology` | `OntologyGenerator` |
46+
| Validate a graph against SHACL constraints | `ontology` | `SHACLGenerator`, `OntologyValidator` |
47+
| Derive new facts from existing knowledge | `reasoning` | `Reasoner`, `GraphReasoner` |
48+
| Export to RDF Turtle, JSON-LD, or N-Triples | `export` | `RDFExporter` |
49+
| Export to Parquet for Spark / BigQuery | `export` | `ParquetExporter` |
50+
| Export for ArangoDB | `export` | `ArangoAQLExporter` |
51+
| Export to Neo4j or Memgraph via Cypher | `export` | `LPGExporter` |
52+
| Visualize a knowledge graph interactively | `visualization` | `KGVisualizer` |
53+
| Run a reproducible multi-step pipeline | `pipeline` | `PipelineBuilder` |
54+
| Use Semantica from Claude Desktop or Cursor | `mcp_server` | `semantica-mcp` |
55+
| Bootstrap a graph from verified seed data | `seed` | `SeedDataManager` |
56+
| Extend Semantica with a custom component | `core` | `PluginRegistry` |
57+
58+
59+
## Goal-by-Goal Starting Points
60+
61+
Pick your goal to see the minimum imports and a working skeleton.
62+
63+
<Tabs>
64+
<Tab title="Build a Knowledge Graph">
65+
Turn documents, web pages, or databases into a structured, queryable graph.
66+
67+
**Pipeline:** `ingest` → `parse` → `semantic_extract` → `kg`
68+
69+
```python
70+
from semantica.ingest import FileIngestor
71+
from semantica.parse import DocumentParser
72+
from semantica.semantic_extract import NERExtractor, RelationExtractor
73+
from semantica.kg import GraphBuilder
74+
75+
sources = FileIngestor().ingest("report.pdf")
76+
parsed = DocumentParser().parse_document("report.pdf")
77+
78+
# No API key required — pattern-based extraction
79+
entities = NERExtractor(method="pattern").extract(parsed)
80+
relationships = RelationExtractor(method="rule").extract(parsed, entities=entities)
81+
82+
graph = GraphBuilder(merge_entities=True).build(
83+
sources=[{"entities": entities, "relationships": relationships}]
84+
)
85+
print(f"{len(graph.nodes)} nodes, {len(graph.edges)} edges")
86+
```
87+
88+
<Tip>
89+
Pass `method="pattern"` to `NERExtractor` for zero-cost, zero-API-key extraction. Switch to `method="llm"` with any of the supported providers for higher recall.
90+
</Tip>
91+
92+
**Next:** [Quickstart →](quickstart) — full pipeline with visualization and export.
93+
</Tab>
94+
95+
<Tab title="Build GraphRAG">
96+
Ground every LLM response in a structured knowledge graph. Every claim links back to a source node.
97+
98+
**Module:** `context`
99+
100+
```python
101+
from semantica.context import AgentContext, ContextGraph
102+
from semantica.vector_store import VectorStore
103+
from semantica.llms import Groq
104+
105+
llm = Groq(model="llama-3.3-70b-versatile")
106+
107+
context = AgentContext(
108+
vector_store=VectorStore(backend="faiss", dimension=768),
109+
knowledge_graph=ContextGraph(advanced_analytics=True),
110+
)
111+
112+
# Store facts — retrieval uses both vectors and graph structure
113+
context.store("Apple Inc. was co-founded by Steve Jobs in 1976 in Cupertino.")
114+
115+
# GraphRAG query with multi-hop reasoning trace
116+
result = context.query_with_reasoning(
117+
"Who co-founded Apple?",
118+
llm_provider=llm,
119+
max_hops=2,
120+
)
121+
print(result["response"]) # grounded answer
122+
print(result["reasoning_path"]) # multi-hop trace
123+
```
124+
125+
**Next:** [Context module reference →](reference/context)
126+
</Tab>
127+
128+
<Tab title="Add Agent Memory">
129+
Give an AI agent persistent memory, decision tracking, and precedent search across sessions.
130+
131+
**Module:** `context`
132+
133+
```python
134+
from semantica.context import AgentContext, ContextGraph
135+
from semantica.vector_store import VectorStore
136+
137+
context = AgentContext(
138+
vector_store=VectorStore(backend="faiss", dimension=768),
139+
knowledge_graph=ContextGraph(advanced_analytics=True),
140+
decision_tracking=True, # required to use record_decision()
141+
)
142+
143+
# Store a memory
144+
context.store("GPT-4 outperforms GPT-3.5 on reasoning benchmarks by 40%.")
145+
146+
# Record a decision with full causal context
147+
decision_id = context.record_decision(
148+
category="model_selection",
149+
scenario="Choose LLM for production reasoning pipeline",
150+
reasoning="GPT-4 benchmark advantage justifies cost increase",
151+
outcome="selected_gpt4",
152+
confidence=0.91,
153+
)
154+
155+
# Search past decisions before making a new one
156+
precedents = context.find_precedents("model selection", limit=5)
157+
158+
# Trace what happened downstream from this decision
159+
chain = context.get_causal_chain(decision_id, direction="downstream")
160+
```
161+
162+
<Note>
163+
`decision_tracking=True` is required. Without it, `record_decision()` raises `RuntimeError`.
164+
</Note>
165+
166+
**Next:** [Context module reference →](reference/context)
167+
</Tab>
168+
169+
<Tab title="Track Provenance">
170+
W3C PROV-O lineage on every fact: source document, extraction method, timestamp, and checksum.
171+
172+
**Modules:** `provenance`, `change_management`
173+
174+
```python
175+
from semantica.provenance import ProvenanceManager
176+
177+
prov = ProvenanceManager()
178+
179+
# Track an entity with full source details
180+
prov.track_entity(
181+
entity_id="entity_1",
182+
source="DOI:10.1371/journal.pone.0023601",
183+
source_location="Figure 2",
184+
confidence=0.92,
185+
)
186+
187+
# Retrieve the complete lineage for this entity
188+
lineage = prov.get_lineage("entity_1")
189+
190+
# Version-control the graph with SHA-256 checksums
191+
from semantica.change_management import TemporalVersionManager
192+
193+
manager = TemporalVersionManager()
194+
snapshot = manager.create_snapshot(kg, "v1.0", "user@example.com", "Initial build")
195+
diff = manager.diff("v1.0", "v1.1")
196+
```
197+
198+
**Next:** [Provenance reference →](reference/provenance) · [Change Management reference →](reference/change_management)
199+
</Tab>
200+
201+
<Tab title="Export">
202+
Serialize your knowledge graph for the semantic web, analytics platforms, or graph databases.
203+
204+
**Module:** `export`
205+
206+
```python
207+
from semantica.export import RDFExporter, ParquetExporter, LPGExporter, ArangoAQLExporter
208+
209+
# RDF — multiple serialization formats
210+
RDFExporter().export(graph, "graph.ttl", format="turtle")
211+
RDFExporter().export(graph, "graph.jsonld", format="jsonld")
212+
213+
# Parquet — for Spark, BigQuery, Databricks, Snowflake
214+
ParquetExporter().export(graph, "output/graph.parquet")
215+
216+
# Neo4j / Memgraph via Cypher
217+
LPGExporter().export(graph, "graph.cypher")
218+
219+
# ArangoDB AQL inserts
220+
ArangoAQLExporter().export(graph, "graph.aql")
221+
```
222+
223+
**Formats:** Turtle · JSON-LD · N-Triples · RDF/XML · Parquet · Cypher · Arrow · OWL · CSV · ArangoDB AQL
224+
225+
**Next:** [Export module reference →](reference/export)
226+
</Tab>
227+
228+
<Tab title="MCP — Claude / Cursor">
229+
Use Semantica from Claude Desktop, Cursor, VS Code, or any MCP-aware tool — no Python code required after setup. 12 tools available instantly.
230+
231+
**Step 1 — Install:**
232+
```bash
233+
pip install semantica
234+
```
235+
236+
**Step 2 — Add to your MCP client config:**
237+
238+
<CodeGroup>
239+
240+
```json Claude Desktop / Windsurf / Cline
241+
{
242+
"mcpServers": {
243+
"semantica": {
244+
"command": "semantica-mcp"
245+
}
246+
}
247+
}
248+
```
249+
250+
```json Cursor / VS Code / Continue
251+
{
252+
"mcpServers": {
253+
"semantica": {
254+
"command": "semantica-mcp",
255+
"env": {
256+
"SEMANTICA_KG_PATH": "/path/to/my_graph.json"
257+
}
258+
}
259+
}
260+
}
261+
```
262+
263+
</CodeGroup>
264+
265+
**Available tools:** `extract_entities` · `extract_relations` · `add_entity` · `add_relationship` · `record_decision` · `query_decisions` · `find_precedents` · `get_causal_chain` · `run_reasoning` · `get_graph_analytics` · `export_graph` · `get_graph_summary`
266+
267+
<Warning>
268+
Set `SEMANTICA_KG_PATH` to persist your graph across restarts. Without it, all data is lost when the server process exits.
269+
</Warning>
270+
271+
**Next:** [MCP Server reference →](reference/mcp_server)
272+
</Tab>
273+
</Tabs>
274+
275+
276+
## Still Unsure?
277+
278+
<AccordionGroup>
279+
<Accordion title="Knowledge graph vs. vector store — which do I need?" icon="scale-balanced">
280+
Use a **knowledge graph** (`kg`) when you need structured reasoning, multi-hop traversal, provenance, or compliance audit trails.
281+
282+
Use a **vector store** (`vector_store`) when you need fast fuzzy similarity search over large text corpora and relationships between items don't matter.
283+
284+
Use **both together** via `AgentContext` (GraphRAG) to get grounded LLM responses where every claim traces back to a source node.
285+
286+
See also: [Core Concepts](concepts)
287+
</Accordion>
288+
289+
<Accordion title="I just want to run something quickly." icon="rocket">
290+
Start with the [Quickstart](quickstart). It builds a complete pipeline (ingest → parse → extract → graph → visualize → export) with no API key required.
291+
</Accordion>
292+
293+
<Accordion title="I'm adding Semantica to an existing agent — what's the minimum?" icon="plug">
294+
Add `AgentContext`. It wraps your existing agent with memory, decision tracking, and precedent search — no changes to your LLM provider or agent framework needed.
295+
296+
```python
297+
from semantica.context import AgentContext, ContextGraph
298+
from semantica.vector_store import VectorStore
299+
300+
context = AgentContext(
301+
vector_store=VectorStore(backend="faiss", dimension=768),
302+
knowledge_graph=ContextGraph(advanced_analytics=True),
303+
decision_tracking=True,
304+
)
305+
```
306+
307+
[Context module reference →](reference/context)
308+
</Accordion>
309+
310+
<Accordion title="I need a compliance-ready pipeline — what's the minimum stack?" icon="shield-check">
311+
| Layer | Module | Key class |
312+
| :---- | :------ | :--------- |
313+
| Ingestion | `ingest` | `FileIngestor` |
314+
| Extraction | `semantic_extract` | `NERExtractor` |
315+
| Graph | `kg` | `GraphBuilder` |
316+
| Lineage | `provenance` | `ProvenanceManager` |
317+
| Versioning | `change_management` | `TemporalVersionManager` |
318+
| Audit export | `export` | `RDFExporter` |
319+
Supports HIPAA, SOX, GDPR, and FDA 21 CFR Part 11 audit requirements.
320+
</Accordion>
321+
</AccordionGroup>
322+
323+
---
324+
325+
- [Quickstart](quickstart) — Full pipeline in 5 minutes.
326+
- [Module Reference](modules) — Every module with examples and common chains.
327+
- [API Reference](reference/context) — Complete class and method documentation.

‎docs/docs.json‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
"pages": [
6666
"concepts",
6767
"modules",
68+
"choose-your-module",
6869
"glossary"
6970
]
7071
},
@@ -104,10 +105,11 @@
104105
"group": "Get Started",
105106
"pages": [
106107
"installation",
107-
"cli-setup",
108-
"explorer-setup",
108+
"getting-started",
109109
"quickstart",
110-
"getting-started"
110+
"choose-your-module",
111+
"cli-setup",
112+
"explorer-setup"
111113
]
112114
}
113115
]

‎docs/getting-started.md‎

Lines changed: 3 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -202,22 +202,9 @@ Semantica uses a modular, layered architecture: import only what you need.
202202
- **[Output Layer](reference/export)** — Deliver results downstream. Modules: `export`, `visualization`, `pipeline`, `explorer`
203203

204204

205-
## "Which module do I need?" Quick Reference
206-
207-
| I want to... | Module | Key class |
208-
| :------------ | :------ | :--------- |
209-
| Load a PDF / web page / database | `ingest` | `FileIngestor`, `WebIngestor` |
210-
| Extract text and tables from a PDF | `parse` | `DocumentParser`, `DoclingParser` |
211-
| Find entities in text | `semantic_extract` | `NERExtractor` |
212-
| Build a knowledge graph | `kg` | `GraphBuilder` |
213-
| Store and search vectors | `vector_store` | `VectorStore` |
214-
| Give my agent persistent memory | `context` | `AgentContext` |
215-
| Record AI decisions with audit trail | `context` | `AgentContext.record_decision()` |
216-
| Query my graph with natural language | `reasoning` | `GraphReasoner` |
217-
| Export to RDF / Neo4j / Parquet | `export` | `RDFExporter`, `LPGExporter` |
218-
| Visualize a knowledge graph | `visualization` | `KGVisualizer` |
219-
| Run a reproducible pipeline | `pipeline` | `PipelineBuilder` |
220-
| Use Semantica from Claude Desktop | `mcp_server` | `semantica-mcp` |
205+
## Which Module Do I Need?
206+
207+
See the [Choose the Right Module](choose-your-module) guide — it maps 35+ developer goals to the right starting point across all 27 modules, with working code for the most common paths.
221208

222209

223210
## Next Steps

‎docs/modules.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,10 @@ icon: "puzzle-piece"
88
Looking for a quick reference? Jump to the [Module Index](#module-index) at the bottom.
99
</Info>
1010

11+
<Tip>
12+
Not sure which module to use? The [Choose the Right Module](choose-your-module) guide maps 35+ developer goals to modules with code examples — start there if you're orienting for the first time.
13+
</Tip>
14+
1115
Semantica is organized into **27 modules** across six logical layers. Each module is independently importable: you never pay for what you don't use.
1216

1317
## Architecture Overview

0 commit comments

Comments
 (0)