Skip to content
This repository was archived by the owner on Feb 18, 2020. It is now read-only.

Fetch entities #7

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions fetch_entities.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
"""
Chron.
Copyright (C) 2018 Alisa Belyaeva, Ata Ali Kilicli, Amaury Martiny,
Daniil Mordasov, Liam O’Flynn, Mikhail Orlov.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
"""

from datetime import datetime
import requests
import os

URL = "https://query.wikidata.org/sparql"
QUERY = """
SELECT ?entity ?predecessors
WHERE
{
VALUES ?countryclass { wd:Q3024240 wd:Q6256 }
?entity wdt:P31 ?countryclass ;
wdt:P571 ?inception .
OPTIONAL { ?entity wdt:P576 ?dissolved } .
OPTIONAL { ?entity wdt:P155 ?predecessors } .
FILTER (?inception < "1815-01-01T00:00:00Z"^^xsd:dateTime)
FILTER (?dissolved >= "1789-01-01T00:00:00Z"^^xsd:dateTime || !Bound(?dissolved) )
SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }
}
"""
R_ENTITIES = requests.get(URL, params={"format": "json", "query": QUERY})
ENTITIES = R_ENTITIES.json()

for entity in ENTITIES["results"]["bindings"]:
data = {
"wikidata_id": int(entity["entity"]["value"].split("Q", 1)[1]),
"color": "#fff", # TODO: change
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are "wish" colors to be assigned manually?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now it's an int field in database.

"admin_level": 1,
"predecessors": [] # entity["predecessors"]["value"]
Copy link
Member Author

@quorth0n quorth0n Jan 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FKEYs are enforced on a db level to maintain integrity, should we change this to be able to store predecessor relationships to entities not in our db?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaik predecessor field was disabled for mvp and primary key has been changed to id
It's better to keep this field as FKEY in the future.

}
r_te = requests.post(
os.getenv("API_ROOT", "http://localhost/api/") + "/territorial-entities/",
data,
params={"format": "json"},
)
print(str(r_te.status_code) + ": " + r_te.reason)