Worksheet Generator Feature PR #102
Conversation
Update App YML for GAE Deployment
worksheet generator Implementation
worksheet generator
Adding hint types and generated QnA evaluation
Adding Worksheet Generator Readme
There was a problem hiding this comment.
Please add description to your PR as well. You can use this pull request as inspiration.
Worksheet Generator Revised README
| - For worksheet generation, use the `create_worksheet_questions()` method. | ||
| 3. The system will generate, validate, and return the questions as a list of dictionaries. Each dictionary contains a question, its answer, and an explanation. | ||
|
|
||
| ## Example Usage |
There was a problem hiding this comment.
Good README.md. You can try adding the request and response interfaces for enabling a better understanding of what the dev team can expect to interact with during the requests to the AI endpoint.
| logger = setup_logger() | ||
|
|
||
|
|
||
| def executor(files: list[ToolFile], topic: str, level: str, hint: str, hint_num: int, num_questions: int, verbose=True): |
There was a problem hiding this comment.
You can use a Pydantic schema for decoupling the args.
| try: | ||
| if verbose: logger.debug(f"Files: {files}") | ||
| # Instantiate RAG pipeline with default values | ||
| pipeline = RAGpipeline(verbose=verbose) |
There was a problem hiding this comment.
Chage the name of this class as it does not support a complete understanding of what it does for the Worksheet Generator.
| db = pipeline(files) | ||
|
|
||
| # Create and return the quiz questions | ||
| output = WorksheetBuilder(db, topic, level, hint, "quiz").create_questions(num_questions) |
There was a problem hiding this comment.
You can instantiate your WorksheetBuilder class first instead for reusing the same variable and improve the readability.
There was a problem hiding this comment.
What's the difference between this file and the other worksheet-generator-quiz-prompt.txt? If they are different, please make sure to use different names for better readability.
| attempts = 0 | ||
| max_attempts = num_questions * 5 # Allow for more attempts to generate questions | ||
| print("len of gen qs", len(generated_questions)) | ||
| while len(generated_questions) < num_questions and attempts < max_attempts: |
There was a problem hiding this comment.
Ensure that this while loop is not costful for optimal results.
|
|
||
| # Return the list of questions | ||
| return generated_questions[:num_questions] | ||
| def validate_qtype_response(self, hint: str, answer) -> bool: |
There was a problem hiding this comment.
This is not required as the JSONOutputParser supports this.
| except ValueError: | ||
| return False | ||
|
|
||
| def is_response_relevant(self, response: dict) -> bool: |
There was a problem hiding this comment.
This proposal is interesting but costful in terms of production resources. Indeed, Google actually provides metrics to accomplish this semantic search process with their embeddings.
| max_attempts = hint_num * 5 | ||
|
|
||
| print("len of gen qs", len(generated_questions)) | ||
| while len(generated_questions) < hint_num and attempts < max_attempts: |
There was a problem hiding this comment.
Ensure that this while loop is not costful in terms of token generation.
| if os.environ['ENV_TYPE'] == "production": | ||
| set_key = access_secret_file("backend-access") | ||
| else: | ||
| set_key = "dev" | ||
|
|
||
| if api_key is None: |
There was a problem hiding this comment.
Please, remove this as it is affecting the auth. of the AI endpoint.
AaronSosaRamos
added
type:enhancement
|
Closed due inactivity |
Worksheet Question Generator
This project enables the generation of quiz and worksheet questions using a machine learning model, integrated with LangChain. Users can provide topics, difficulty levels, and hints to generate customized questions, with robust validation processes to ensure accuracy and relevance.
Key Features
Dual-Prompt Setup
Based on experimentation, using two distinct prompts—one for quiz questions and another for worksheet questions—yields optimal results. This dual-prompt approach enhances the quality and relevance of the generated content.
Question Generation
The
WorksheetBuilderclass supports the generation of both quiz and worksheet questions based on:Validation Mechanism
Logging & Error Handling
Detailed logs at key stages ensure smooth execution and help debug any issues.
How to Use
WorksheetBuilderclass with the required parameters.create_questions()for quiz questions orcreate_worksheet_questions()for worksheet questions.Example