Initialization

[markwhiting]

• Import prior statements
• Add in scripts for cleaning
• Add in scripts for generating
• Maintain pipelines where possible

[amirrr]

The prompt used for statement labeling is as follows:

You are a bot that takes a statement from the user and provides properties of the statement based on the following definitions. Please don't explain your answer and provide it in JSON format:

{
    "behavior": ,
    "everyday": ,
    "figure_of_speech": ,
    "judgment": ,
    "opinion": ,
    "reasoning": ,
    "knowledge category": 
}

Dimensions of a statement and their definitions:

Is it behavior?

• yes: It refers to beliefs, perceptions, preferences, and socially constructed rules that govern human experience; it can be “real” or opinion, but is intrinsically of human origins.
  • e.g., I exist and am the same person I was yesterday. He yelled at me because he was angry. There are seven days in the week.
• no: It refers to objective features of the world as described by, say, physics, biology, engineering, mathematics, or other natural rules; it can be measured empirically or derived logically.
  • e.g., Men on average are taller than women. The Earth is the third planet from the Sun. Ants are smaller than Elephants.

Is it everyday?

• yes: People encounter, or could encounter, situations like this in the course of their ordinary, everyday experiences.
  • e.g., Touching a hot stove will burn you. Commuting at rush hour takes longer. It is rude to jump the line.
• no: This claim refers to regularities or conclusions that cannot be observed or arrived at solely through individual experience.
  • e.g., Capitalism is a better economic system than Communism. Strict gun laws save lives. God exists.

Is it figure_of_speech?

• yes: It contains an aphorism, metaphor, hyperbole.
  • e.g., Birds of a feather flock together. A friend to all is a friend to none.
• no: It is plain and ordinary language that means exactly what it says.
  • e.g., The sky is blue. Elephants are larger than dogs. Abraham Lincoln was a great president.

Is it judgment?

• yes: It refers to a judgment, belief, value, social norm, or convention.
  • e.g., If you are going to the office, you should wear business attire, not a bathing suit. Treat others how you want them to treat you. Freedom is a fundamental human right.
• no: It refers to something in the world, such as an empirical regularity or scientific law.
  • e.g., hot things will burn you; the sun rises in the east and sets in the west.

Is it opinion?

• yes: It is something that someone might think is true, or wants others to think is true, but can’t be demonstrated to be objectively correct or incorrect; it is inherently subjective.
  • e.g., FDR was the greatest US president of the 20th Century. The Brooklyn Bridge is prettier than the Golden Gate. Vaccine mandates are a tolerable imposition on individual freedom.
• no: It is something that can be demonstrated to be correct or incorrect, independently of anyone’s opinion.
  • e.g., the earth is the third planet from the sun (this is correct and we know it is correct), Obama was the 24th president of the United States (this is incorrect, but we know it’s incorrect). It will be sunny next Tuesday (we don’t yet know if this is correct, but we will be able to check in the future).

Is it reasoning?

• yes: The claim presents a conclusion that is arrived at by combining knowledge and logic.
  • e.g., The sun is in the east, therefore it is morning. My dog is wagging its tail, therefore it is happy. The glass fell off the table, therefore it will break and the floor will become wet.
• no: The claim refers to some observation about the world; it may be true or false, opinion or fact, subjective or objective.
  • e.g., The sun rises in the east and sets in the west. Dogs are nicer than cats. Glasses break when they are dropped.

Which knowledge category or categories describe this claim? (choose one that apply)

• General reference
• Culture and the arts
• Geography and places
• Health and fitness
• History and events
• Human activities
• Mathematics and logic
• Natural and physical sciences
• People and self
• Philosophy and thinking
• Religion and belief systems
• Society and social sciences
• Technology and applied sciences

[markwhiting]

We should have a few statements CSVs in this repo + any needed actions and related code, but basically nothing else.

For CSVs

statements.csv: the CSV has a single line for each statement, with an id and a parent id, features, metadata columns (e.g., in_use).
statement_embeddings.: a reasonably accessible format that maintains precision

For code

/scripts
prompts.json
clean.py
rate.py
rating_weights.json

Notes

Where to save one off scripts? E.G., generation