Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use full combined results for student evaluation on asynchronous activities #4341

Merged
merged 2 commits into from
Oct 29, 2024

Conversation

sjschlapbach
Copy link
Member

@sjschlapbach sjschlapbach commented Oct 29, 2024

Summary by CodeRabbit

Release Notes

  • New Features

    • Introduced structured response handling for evaluations with new types: SingleChoiceResponse, SingleNumericalResponse, and SingleFreeTextResponse.
    • Enhanced feedback evaluations by including response counts alongside values in both numerical and free-text evaluations.
  • Improvements

    • Streamlined data handling in evaluation components, directly accessing updated response structures.
    • Updated GraphQL schema to reflect new response formats, improving clarity and usability in queries.
    • Enhanced evaluation logic to incorporate anonymous results for more comprehensive response aggregation.
  • Bug Fixes

    • Resolved issues with data transformation in evaluation components, ensuring accurate rendering of responses.

Copy link

coderabbitai bot commented Oct 29, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

This pull request introduces significant modifications to the GraphQL schema and related components, primarily focused on restructuring the answers and responses fields in NumericalInstanceEvaluation and FreeTextInstanceEvaluation types. The updates include the introduction of new response types, changes to existing types, and adjustments in various components and services to accommodate these schema changes. The overall goal is to enhance the clarity and structure of response handling for both numerical and free-text evaluations.

Changes

File Path Change Summary
packages/graphql/src/graphql/ops/FStackFeedbackEvaluations.graphql Updated answers field to responses in NumericalInstanceEvaluation and FreeTextInstanceEvaluation.
packages/graphql/src/ops.schema.json Added new types (SingleFreeTextResponse, SingleNumericalResponse, NumericalSolutionRange), modified NumericalInstanceEvaluation and StackResponseInput.
packages/graphql/src/public/client.json Updated hashes for RespondToElementStack and GetPreviousStackEvaluation.
packages/graphql/src/public/schema.graphql Changed answers to responses in NumericalInstanceEvaluation and FreeTextInstanceEvaluation, added new response types.
packages/graphql/src/public/server.json Updated mutations and fragments to include new responses field in evaluations.
packages/graphql/src/schema/evaluation.ts Added IStackFeedback interface and new imports, enhancing type definitions.
packages/graphql/src/schema/practiceQuizzes.ts Removed IStackFeedback interface and updated StackFeedback object reference.
packages/graphql/src/schema/question.ts Added new response types and updated responses field in evaluations.
packages/graphql/src/services/stacks.ts Refactored evaluation functions to include anonymous results.
packages/shared-components/src/evaluation/FTEvaluation.tsx Simplified handling of evaluation.answers in rendering logic.
packages/shared-components/src/evaluation/NREvaluation.tsx Simplified handling of evaluation.responses in rendering logic.
packages/types/src/index.ts Added new types for responses and updated interfaces for evaluations.

Possibly related PRs

Suggested reviewers

  • rschlaefli: Suggested reviewer for the changes made in this pull request.

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between e1e0c0f and 68d5c5e.

📒 Files selected for processing (10)
  • packages/graphql/src/graphql/ops/FStackFeedbackEvaluations.graphql (3 hunks)
  • packages/graphql/src/ops.schema.json (4 hunks)
  • packages/graphql/src/public/client.json (2 hunks)
  • packages/graphql/src/public/schema.graphql (4 hunks)
  • packages/graphql/src/public/server.json (2 hunks)
  • packages/graphql/src/schema/question.ts (3 hunks)
  • packages/graphql/src/services/stacks.ts (15 hunks)
  • packages/shared-components/src/evaluation/MCKPRIMEvaluation.tsx (1 hunks)
  • packages/shared-components/src/evaluation/SCEvaluation.tsx (1 hunks)
  • packages/types/src/index.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • packages/graphql/src/graphql/ops/FStackFeedbackEvaluations.graphql
  • packages/graphql/src/public/client.json
🔇 Additional comments (11)
packages/shared-components/src/evaluation/MCKPRIMEvaluation.tsx (1)

20-42: LGTM! The refactoring improves type safety and readability.

The transition from Object.entries to direct array mapping with structured choice objects is a good improvement. The new implementation:

  • Provides better type safety through structured data
  • Makes the code more maintainable by eliminating the need for type casting
  • Maintains the correct functionality for progress calculation and styling

Let's verify that the new choice structure is consistently used across the codebase:

packages/graphql/src/schema/question.ts (1)

175-192: LGTM: Well-structured choice response type

The implementation of SingleChoiceResponse and its integration with ChoicesInstanceEvaluation is clean and follows the schema design patterns.

packages/graphql/src/public/schema.graphql (1)

150-150: Verify the impact on existing queries.

The schema changes from JSON to structured types might affect existing queries.

Also applies to: 665-665, 1132-1132

✅ Verification successful

Schema changes appear to be compatible with existing queries

Based on the search results, the schema changes are safe as the existing queries are already using structured types:

  • The choices field is used consistently in queries with proper field selection, e.g.:
choices {
  ix
  correct
}
  • The responses field is used with proper field selection in queries like:
responses {
  id
  content
}

The schema changes maintain backward compatibility with these existing query patterns.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for GraphQL queries/mutations using the modified fields
# Description: Look for queries that might need updates due to schema changes

# Check for queries accessing choices field
rg -t graphql "choices\s*{" 

# Check for queries accessing answers field
rg -t graphql "answers\s*{" 

# Check for queries accessing responses field
rg -t graphql "responses\s*{" 

Length of output: 353


Script:

#!/bin/bash
# Search for GraphQL queries/mutations using the modified fields
# Description: Look for queries that might need updates due to schema changes

# Check for queries accessing choices field
rg -t graphql "choices" -A 2

# Check for queries accessing answers field
rg -t graphql "answers" -A 2

# Check for queries accessing responses field
rg -t graphql "responses" -A 2

Length of output: 184


Script:

#!/bin/bash
# Search for GraphQL queries/mutations using the modified fields
# Description: Look for queries that might need updates due to schema changes

# Find all GraphQL files first
fd "\.(graphql|gql)$" | while read -r file; do
  echo "=== Checking $file ==="
  # Check for queries accessing choices field
  rg "choices" "$file" -A 2 || true
  # Check for queries accessing answers field
  rg "answers" "$file" -A 2 || true
  # Check for queries accessing responses field
  rg "responses" "$file" -A 2 || true
done

Length of output: 86799

packages/graphql/src/services/stacks.ts (2)

1134-1134: LGTM: Type definition is well-structured

The type definition for SharedEvaluationProps | 'solutionRanges' | 'responses' is clear and follows TypeScript best practices.


1170-1202: LGTM: Evaluation function properly combines results

The function correctly combines participant and anonymous results for choices evaluation.

packages/graphql/src/public/server.json (2)

142-142: Skipping review of duplicate fragment.

This is a duplicate of the StackFeedbackEvaluations fragment reviewed above.


83-83: LGTM: Response handling improvements in StackFeedbackEvaluations fragment.

The changes to restructure response handling in evaluation types look good. The fragment now consistently tracks response counts across different question types, improving data consistency and analysis capabilities.

Let's verify the fragment usage across the codebase:

✅ Verification successful

The search results show that the StackFeedbackEvaluations fragment appears in three locations:

  1. In the server.json file with hash "6d6a9d682fec..." (RespondToElementStack mutation)
  2. In the server.json file with hash "82270825f35e..." (GetPreviousStackEvaluation query)
  3. In the source file packages/graphql/src/graphql/ops/FStackFeedbackEvaluations.graphql

Let's verify if all these instances are consistent:


Response handling structure is consistent across all usages

The verification confirms that the StackFeedbackEvaluations fragment implementation is consistent between the source file and its compiled versions in server.json. The response handling structure is uniform across all evaluation types:

  • ChoicesInstanceEvaluation uses choices with count
  • NumericalInstanceEvaluation uses responses with value and count
  • FreeTextInstanceEvaluation uses answers with value and count
  • FlashcardInstanceEvaluation and ContentInstanceEvaluation maintain their specific response formats
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all usages of StackFeedbackEvaluations fragment to ensure consistent implementation
# Test: Search for fragment usage. Expect: Only occurrences with the new response structure.

rg -A 5 "fragment StackFeedbackEvaluations"

Length of output: 15072


Script:

#!/bin/bash
# Get the full content of the source fragment file to compare with the compiled versions
cat packages/graphql/src/graphql/ops/FStackFeedbackEvaluations.graphql

Length of output: 2520

packages/graphql/src/ops.schema.json (3)

1663-1673: LGTM: Well-structured type definition for choice responses

The non-null list type definition for SingleChoiceResponse follows GraphQL schema best practices and ensures type safety.


7496-7506: LGTM: Consistent type definition for free text responses

The type definition maintains consistency with other response types and ensures proper type safety.


17312-17331: Verify the migration from 'answers' to 'responses' field

The addition of the 'responses' field with SingleNumericalResponse type aligns with the PR objective of improving student evaluation. However, we should verify that all consumers of the old 'answers' field have been updated.

✅ Verification successful

Looking at the search results, I can see that in the TypeScript definitions in ops.ts, the NumericalInstanceEvaluation type already uses the new responses field and not the old answers field. This is consistent across all three type definitions (StackFeedbackEvaluationsFragment, RespondToElementStackMutation, and GetPreviousStackEvaluationQuery).

Let me verify if there are any other references to the old field by searching with a broader pattern:


Let me search for any GraphQL schema definitions to get a complete picture:


Let me try a different approach to search the schema files:


Based on all the search results, I can now provide a final response:

Migration from 'answers' to 'responses' field is correctly implemented

The schema definition and all GraphQL operations consistently use the new 'responses' field in NumericalInstanceEvaluation type. There are no remaining references to an 'answers' field for this type, confirming the migration is complete and consistent across the codebase.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for any remaining references to the old 'answers' field in GraphQL queries/mutations
rg -t graphql 'answers.*NumericalInstanceEvaluation' 

# Search for any TypeScript/JavaScript files that might need updates
rg -t ts -t js 'answers.*NumericalInstanceEvaluation'

Length of output: 11287


Script:

#!/bin/bash
# Search for any GraphQL schema files
fd schema.graphql

# Search for any remaining references to 'answers' field specifically in NumericalInstanceEvaluation
rg "NumericalInstanceEvaluation.*answers"
rg "answers.*NumericalInstanceEvaluation"

Length of output: 49053


Script:

#!/bin/bash
# Search for NumericalInstanceEvaluation type definition in schema files
rg -t graphql -t gql "type NumericalInstanceEvaluation"

# Also check for any remaining references to 'answers' in schema files
rg -t graphql -t gql "answers.*NumericalInstanceEvaluation|NumericalInstanceEvaluation.*answers"

Length of output: 220


Script:

#!/bin/bash
# Find and search schema files without relying on file type
fd "schema\.(graphql|gql)" -x rg -l "type NumericalInstanceEvaluation" {}

# Search in the specific schema file we know exists
cat packages/graphql/src/public/schema.graphql | grep -A 20 "type NumericalInstanceEvaluation"

Length of output: 706

packages/types/src/index.ts (1)

457-462: Ensure consistent optionality of responses property

In IInstanceEvaluationNumerical, the responses property is optional (responses?), while in IInstanceEvaluationChoices, the choices (or responses if renamed) property is required. Verify whether the responses property should be required in IInstanceEvaluationNumerical for consistency.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

cypress bot commented Oct 29, 2024

klicker-uzh    Run #3445

Run Properties:  status check passed Passed #3445  •  git commit 9a54f78abf: fix: migrate numerical question evaluation on asynchronous activities to new his...
Project klicker-uzh
Branch Review v3-new-live-quiz
Run status status check passed Passed #3445
Run duration 11m 17s
Commit git commit 9a54f78abf: fix: migrate numerical question evaluation on asynchronous activities to new his...
Committer Julius Schlapbach
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 140
View all changes introduced in this branch ↗︎

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments.

Actionable comments posted: 8

🧹 Outside diff range and nitpick comments (5)
packages/shared-components/src/evaluation/NREvaluation.tsx (1)

Line range hint 1-41: Consider adding TypeScript type for responses.

While the component works correctly, it would be beneficial to explicitly type the shape of the response data for better type safety and documentation.

Consider adding a type definition:

interface NumericalResponse {
  value: number
  count: number
}

And then update the prop types:

 interface NREvaluationProps {
   options: NumericalQuestionOptions
-  evaluation: NumericalInstanceEvaluation
+  evaluation: NumericalInstanceEvaluation & {
+    responses: NumericalResponse[]
+  }
   reference?: string
 }
packages/graphql/src/schema/practiceQuizzes.ts (1)

Line range hint 71-84: Update StackFeedback implementation to remove IStackFeedback reference.

The StackFeedback object reference still uses the IStackFeedback type which is being removed. The implementation should be updated to directly define the type structure.

Apply this diff to update the implementation:

-export const StackFeedback = builder
-  .objectRef<IStackFeedback>('StackFeedback')
-  .implement({
+export const StackFeedback = builder
+  .objectRef('StackFeedback')
+  .implement({
    fields: (t) => ({
      id: t.exposeInt('id'),
      status: t.expose('status', { type: StackFeedbackStatus }),
      score: t.exposeInt('score', { nullable: true }),
      evaluations: t.expose('evaluations', {
        type: [InstanceEvaluation],
        nullable: true,
      }),
    }),
  })
packages/graphql/src/schema/evaluation.ts (1)

19-24: Add JSDoc documentation for the new interface.

Consider adding documentation to explain the purpose and usage of this interface, especially since it's part of a student evaluation fix.

Example documentation:

+/**
+ * Represents feedback for a stack of questions in student evaluation.
+ * @property {number} id - Unique identifier for the stack feedback
+ * @property {StackFeedbackStatusType} status - Current status of the feedback
+ * @property {number} [score] - Optional numerical score for the stack
+ * @property {IInstanceEvaluation[]} [evaluations] - Optional array of instance evaluations
+ */
 export interface IStackFeedback {
packages/graphql/src/ops.schema.json (1)

Line range hint 7488-22603: Schema changes improve response aggregation capabilities.

The restructuring from 'answers' to 'responses' with the new types provides better support for aggregating student responses in asynchronous activities. The consistent pattern between numerical and text responses makes the API more intuitive.

Consider these operational aspects:

  1. Ensure proper migration of existing data to the new structure
  2. Update client applications to handle the new response format
  3. Consider adding schema deprecation notices for the old 'answers' field if still present
packages/graphql/src/services/stacks.ts (1)

2228-2232: Possible redundancy due to multiple evaluations

Consider removing duplicate evaluation calls if they are redundant. Ensure that the evaluation logic is streamlined to prevent unnecessary computations.

[duplication_comment]

🛑 Comments failed to post (8)
packages/graphql/src/schema/practiceQuizzes.ts (1)

8-8: ⚠️ Potential issue

Remove unused import of IStackFeedback.

According to the changes, the IStackFeedback interface has been removed, but it's still being imported here. This import should be removed as part of the interface removal changes.

Apply this diff to remove the unused import:

-import { IStackFeedback } from './evaluation.js'
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.


packages/graphql/src/schema/evaluation.ts (1)

10-10: 💡 Codebase verification

Move evaluation interfaces to a dedicated types file

The codebase already has an established types directory at packages/graphql/src/types/ and contains non-GraphQL type definitions. The evaluation interfaces in this file are pure TypeScript interfaces that don't use GraphQL types, making them suitable for extraction.

  • Move the following interfaces to a new file packages/graphql/src/types/evaluation.ts:
    • IActivityEvaluation
    • IStackFeedback
    • IStackEvaluation
    • IElementInstanceEvaluation
    • InstanceEvaluationResults
    • All element-specific evaluation interfaces (IChoicesElementEvaluationResults, etc.)
🔗 Analysis chain

Consider addressing the TODO comment.

The TODO comment suggests moving types to a separate file. Since we're adding new types, this might be a good time to implement that suggestion.

Let's check if there are other type definition files in the project:


Let me gather more specific information about the types in the evaluation schema and check for any established patterns in the GraphQL package:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Look for existing type definition files and potential patterns
# to determine the best location for these types

# Check for existing type definition files
echo "Searching for type definition files..."
fd -e ts -e d.ts "types?" --type f

# Check for other GraphQL schema files
echo -e "\nSearching for other GraphQL schema files..."
fd -e ts "schema" --type f

Length of output: 487


Script:

#!/bin/bash
# Check the structure of the GraphQL package
echo "Checking GraphQL package structure..."
fd . packages/graphql/src --type d

# Look for any type definitions in the GraphQL package
echo -e "\nLooking for type-related files in GraphQL package..."
rg -l "type\s+\w+\s*=" packages/graphql/src

# Check the evaluation.ts file content to understand the types
echo -e "\nChecking evaluation.ts content..."
cat packages/graphql/src/schema/evaluation.ts

Length of output: 12768

packages/types/src/index.ts (1)

466-468: ⚠️ Potential issue

Maintain consistency: Rename 'answers' to 'responses' for free text evaluation.

For consistency with IInstanceEvaluationNumerical, consider renaming the answers field to responses. This would make the interface patterns more uniform and easier to maintain.

Apply this change:

 export interface IInstanceEvaluationFreeText extends IBaseInstanceEvaluation {
-  answers?: SingleFreeTextResponse[]
+  responses?: SingleFreeTextResponse[]
   solutions: string[]
   lastResponse?: SingleQuestionResponseValue | null
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

  responses?: SingleFreeTextResponse[]
  solutions: string[]
  lastResponse?: SingleQuestionResponseValue | null
packages/graphql/src/schema/question.ts (2)

9-10: ⚠️ Potential issue

Fix typo in type alias.

There's a typo in the alias name: SingleNumericalRepsonseType is missing an 's'.

-  SingleNumericalResponse as SingleNumericalRepsonseType,
+  SingleNumericalResponse as SingleNumericalResponseType,
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

  SingleFreeTextResponse as SingleFreeTextResponseType,
  SingleNumericalResponse as SingleNumericalResponseType,

188-195: 💡 Codebase verification

⚠️ Potential issue

Based on the search results, I can generate the final response as we have found all the necessary information about the type usage:

Fix typo in SingleNumericalResponse type name and its import.

The type name has a typo in both the import and type definition:

  • The import from @klicker-uzh/types is aliased with a typo as SingleNumericalRepsonseType
  • The type is correctly spelled as SingleNumericalResponse in the builder reference

The type is used correctly in other parts of the codebase (e.g., in ops.ts), so this is just a local issue in the schema definition.

🔗 Analysis chain

Fix typo in SingleNumericalResponse type name.

The type name contains a typo: SingleNumericalRepsonse is missing an 's'.

-export const SingleNumericalRepsonse = builder
+export const SingleNumericalResponse = builder

Let's verify the type usage across the codebase:


The previous search might have failed due to case sensitivity or partial matches. Let's try a broader search pattern to catch both the correct and incorrect spellings, along with examining the full context of the schema file to understand the type usage better.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for any instances of the misspelled type
rg "SingleNumericalRepsonse" --type ts

Length of output: 397


Script:

#!/bin/bash
# Search for any instances of the type name (both correct and incorrect spellings)
rg "SingleNumerical[Rr]e[sp]ponse" --type ts -A 2

# Search for type definitions and references in the schema
fd "question.ts" --exec cat {}

Length of output: 57555

packages/graphql/src/services/stacks.ts (3)

2228-2232: ⚠️ Potential issue

Remove outdated TODO comment

The TODO comment at line 2228~ is outdated, as computeQuestionEvaluation already calls the appropriate evaluation functions (evaluateChoicesElementResponse, evaluateNumericalElementResponse, evaluateFreeTextElementResponse). Please remove or update the TODO comment to reflect the current implementation.


2231-2234: ⚠️ Potential issue

Handle potential undefined options in updatedInstance

When accessing updatedInstance.options.pointsMultiplier, ensure that updatedInstance.options is not undefined to prevent potential runtime errors.

Apply this diff to safely access pointsMultiplier:

 const questionEval = computeQuestionEvaluation({
   elementData: existingInstance.elementData,
   results: updatedInstance.results,
   anonymousResults: updatedInstance.anonymousResults,
-  multiplier: updatedInstance.options.pointsMultiplier,
+  multiplier: updatedInstance.options?.pointsMultiplier,
 })

Committable suggestion was skipped due to low confidence.


1170-1213: 🛠️ Refactor suggestion

Set default value for multiplier in evaluation functions

To ensure consistent scoring and avoid potential undefined values, consider providing a default value for the multiplier parameter in evaluateChoicesElementResponse.

Apply this diff to set a default value:

 function evaluateChoicesElementResponse({
   elementData,
   results,
   anonymousResults,
   correctness,
-  multiplier,
+  multiplier = 1,
 }: {
   elementData: ChoicesElementData
   results: ElementResultsChoices
   anonymousResults: ElementResultsChoices
   correctness: number | null
   multiplier?: number
 }): ChoicesEvaluationReturnType | null {
   // ...
-    pointsMultiplier: multiplier ?? 1,
+    pointsMultiplier: multiplier,

Repeat similar changes for evaluateNumericalElementResponse and evaluateFreeTextElementResponse to maintain consistency.

Committable suggestion was skipped due to low confidence.

Copy link

cypress bot commented Oct 29, 2024

klicker-uzh    Run #3447

Run Properties:  status check passed Passed #3447  •  git commit 0ec60f5cdd ℹ️: Merge 68d5c5e0c760ff522e2372216ce3442b44cb6af2 into 9a54f78abf64a1ccfbdc2aea6b96...
Project klicker-uzh
Branch Review aggregate-results-evaluation
Run status status check passed Passed #3447
Run duration 11m 18s
Commit git commit 0ec60f5cdd ℹ️: Merge 68d5c5e0c760ff522e2372216ce3442b44cb6af2 into 9a54f78abf64a1ccfbdc2aea6b96...
Committer Julius Schlapbach
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 140
View all changes introduced in this branch ↗︎

Copy link

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments.

Actionable comments posted: 5

🧹 Outside diff range and nitpick comments (9)
packages/shared-components/src/evaluation/SCEvaluation.tsx (2)

18-36: Add null checks and handle edge cases.

The changes look good overall, moving to an array-based structure improves code clarity. However, consider adding some safety improvements:

Consider applying these improvements:

- {evaluation.choices?.map((choice) => (
+ {evaluation.choices && evaluation.choices.length > 0 ? (
+   evaluation.choices.map((choice) => (
    <Progress
      isMaxVisible={false}
      className={{
        root: twMerge(
          'h-8',
          choice.ix == correctIx && 'font-bold text-green-600'
        ),
        indicator: twMerge(
          'h-8',
          choice.ix == correctIx ? 'bg-green-600' : 'bg-gray-400'
        ),
      }}
      key={`choice-statistic-${choice.ix}-${choice.count}`}
-     value={(choice.count / (evaluation.numAnswers ?? 1)) * 100}
+     value={evaluation.numAnswers > 0 ? (choice.count / evaluation.numAnswers) * 100 : 0}
      max={100}
      formatter={(v) => v.toFixed() + '%'}
    />
-  ))}
+  ))
+ ) : (
+   <div className="text-gray-500">{t('common.noResponses')}</div>
+ )}

21-30: Optimize className construction.

The current implementation recalculates the className on every render. Consider memoizing these values or moving them outside the component.

+ const getClassNames = (isCorrect: boolean) => ({
+   root: twMerge('h-8', isCorrect && 'font-bold text-green-600'),
+   indicator: twMerge('h-8', isCorrect ? 'bg-green-600' : 'bg-gray-400'),
+ });

function SCEvaluation({ evaluation }: Props) {
  // ... existing code ...
  
  return (
    <div className="space-y-2">
      {evaluation.choices?.map((choice) => (
        <Progress
          isMaxVisible={false}
-         className={{
-           root: twMerge(
-             'h-8',
-             choice.ix == correctIx && 'font-bold text-green-600'
-           ),
-           indicator: twMerge(
-             'h-8',
-             choice.ix == correctIx ? 'bg-green-600' : 'bg-gray-400'
-           ),
-         }}
+         className={getClassNames(choice.ix === correctIx)}
packages/graphql/src/schema/question.ts (1)

GraphQL schema and fragment updates needed

The verification has identified inconsistencies between the schema and its usage:

  1. In FStackFeedbackEvaluations.graphql, FreeTextInstanceEvaluation still uses the old field name answers while the schema defines it as answers.
  2. NumericalInstanceEvaluation correctly uses the responses field.
  3. ChoicesInstanceEvaluation correctly uses the choices field.

Please update the following:

  • In FStackFeedbackEvaluations.graphql, change answers to responses in the FreeTextInstanceEvaluation fragment to match the schema definition.
🔗 Analysis chain

Line range hint 175-246: Verify GraphQL query/fragment updates

Please ensure all GraphQL queries and fragments using these evaluation types are updated to reflect the schema changes:

  1. New response types
  2. Field name changes from answers to responses
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Find all GraphQL files that might need updates
echo "Files potentially requiring updates:"
fd -e graphql -e gql

# Check for potentially outdated field usage
echo -e "\nPotential instances of old field names:"
rg "answers.*FreeTextInstanceEvaluation" --type graphql

Length of output: 11982


Script:

#!/bin/bash
# Search for GraphQL fragments and queries that use these evaluation types
echo "Checking for usage of evaluation types in GraphQL files:"

# Check for FreeTextInstanceEvaluation field usage
echo -e "\nFreeTextInstanceEvaluation field usage:"
rg -A 3 "FreeTextInstanceEvaluation.*{" packages/graphql/src/graphql/ops/

# Check for NumericalInstanceEvaluation field usage
echo -e "\nNumericalInstanceEvaluation field usage:"
rg -A 3 "NumericalInstanceEvaluation.*{" packages/graphql/src/graphql/ops/

# Check for ChoicesInstanceEvaluation field usage
echo -e "\nChoicesInstanceEvaluation field usage:"
rg -A 3 "ChoicesInstanceEvaluation.*{" packages/graphql/src/graphql/ops/

# Check for any references to 'answers' field
echo -e "\nChecking for 'answers' field usage:"
rg "answers.*{" packages/graphql/src/graphql/ops/

Length of output: 1230


Script:

#!/bin/bash
# Search for GraphQL fragments and queries that use these evaluation types
echo "Checking for usage of evaluation types in GraphQL files:"

# Check for FreeTextInstanceEvaluation field usage
echo -e "\nFreeTextInstanceEvaluation field usage:"
rg "FreeTextInstanceEvaluation \{" packages/graphql/src/graphql/ops/

# Check for field names in fragments
echo -e "\nChecking field usage in fragments:"
rg "answers: \[" packages/graphql/src/graphql/ops/
rg "responses: \[" packages/graphql/src/graphql/ops/
rg "choices: \[" packages/graphql/src/graphql/ops/

# Check fragment definitions
echo -e "\nChecking fragment definitions:"
rg "fragment.*InstanceEvaluation" packages/graphql/src/graphql/ops/

Length of output: 997


Script:

#!/bin/bash
# Let's check the content of the file that uses FreeTextInstanceEvaluation
echo "Content of FStackFeedbackEvaluations.graphql:"
cat packages/graphql/src/graphql/ops/FStackFeedbackEvaluations.graphql

# Also check for any inline fragments on these evaluation types
echo -e "\nChecking for inline fragments on evaluation types:"
rg -A 5 "on (FreeTextInstanceEvaluation|NumericalInstanceEvaluation|ChoicesInstanceEvaluation)" packages/graphql/src/graphql/ops/

Length of output: 4482

packages/graphql/src/public/schema.graphql (2)

1515-1518: LGTM! Well-structured response types.

The introduction of explicit response types (SingleChoiceResponse, SingleFreeTextResponse, SingleNumericalResponse) improves type safety and schema clarity. Each type appropriately captures its specific requirements:

  • Choice responses include count and index
  • Free text responses include count and string value
  • Numerical responses include count and float value

This standardization of response types:

  1. Makes the schema more maintainable
  2. Provides better type safety compared to using JSON
  3. Makes it easier to validate responses on the client side
  4. Improves documentation through self-describing types

Also applies to: 1520-1523, 1525-1528


150-150: LGTM! Consistent field updates across evaluation types.

The changes to evaluation types show consistent handling of responses:

  • ChoicesInstanceEvaluation.choices: [SingleChoiceResponse!]
  • FreeTextInstanceEvaluation.answers: [SingleFreeTextResponse!]
  • NumericalInstanceEvaluation.responses: [SingleNumericalResponse!]

The consistent naming and structure across different question types will make it easier to:

  1. Implement generic handling of responses in the frontend
  2. Add new question types in the future
  3. Maintain type safety throughout the application

Also applies to: 665-665, 1132-1132

packages/graphql/src/services/stacks.ts (2)

Line range hint 2667-2749: Consider optimizing numerical results combination

The numerical results combination could be optimized by using a Map to store values instead of array.find operations, which would improve performance for large datasets.

Consider this optimization:

function combineNumericalResults({
  results,
  anonymousResults,
}: {
  results: ElementResultsOpen
  anonymousResults: ElementResultsOpen
}) {
-  return [
-    ...Object.values(results.responses),
-    ...Object.values(anonymousResults.responses),
-  ].reduce<{ value: number; count: number; correct?: boolean | null }[]>(
-    (acc, response) => {
-      const responseValue = parseFloat(response.value)
-      const ix = acc.findIndex(
-        (r) => Math.abs(r.value - responseValue) < Number.EPSILON
-      )
-      if (ix === -1) {
-        acc.push({
-          value: responseValue,
-          count: response.count,
-          correct: response.correct,
-        })
-      } else {
-        acc[ix] = {
-          ...acc[ix]!,
-          count: acc[ix]!.count + response.count,
-        }
-      }
-      return acc
-    },
-    []
-  )
+  const combinedMap = new Map<number, { count: number; correct?: boolean | null }>()
+  
+  const processResponses = (responses: ElementResultsOpen['responses']) => {
+    Object.values(responses).forEach((response) => {
+      const value = parseFloat(response.value)
+      const existing = combinedMap.get(value)
+      if (existing) {
+        existing.count += response.count
+      } else {
+        combinedMap.set(value, { 
+          count: response.count, 
+          correct: response.correct 
+        })
+      }
+    })
+  }
+
+  processResponses(results.responses)
+  processResponses(anonymousResults.responses)
+
+  return Array.from(combinedMap.entries()).map(([value, data]) => ({
+    value,
+    count: data.count,
+    correct: data.correct
+  }))
}

2802-2802: TODO: Implement missing statistics functionality

The comment indicates that statistical analysis features need to be added to numerical responses.

Would you like me to help implement basic statistical measures (mean, median, mode, standard deviation) for numerical responses? I can create a new GitHub issue to track this enhancement.

packages/graphql/src/ops.schema.json (2)

22524-22655: Well-structured response types, consider adding descriptions

The new response types are well-designed with appropriate field types and a consistent pattern. However, they lack descriptions that would help developers understand their purpose and usage.

Consider adding descriptions to the types and fields. For example:

 {
   "kind": "OBJECT",
   "name": "SingleNumericalResponse",
-  "description": null,
+  "description": "Represents an aggregated numerical response with its occurrence count",
   "isOneOf": null,
   "fields": [
     {
       "name": "count",
-      "description": null,
+      "description": "Number of times this response value was submitted",

Line range hint 1663-22655: Strong type modeling for response handling

The restructuring of response types provides several architectural benefits:

  1. Better type safety through specific response types
  2. Consistent pattern across different response kinds
  3. Support for response aggregation through the count field
  4. Clear separation of concerns between different response types

This will make the schema more maintainable and easier to extend in the future.

🛑 Comments failed to post (5)
packages/graphql/src/schema/question.ts (3)

9-11: ⚠️ Potential issue

Fix typo in type name

There's a typo in the imported type name: SingleNumericalRepsonseType should be SingleNumericalResponseType.

-  SingleNumericalResponse as SingleNumericalRepsonseType,
+  SingleNumericalResponse as SingleNumericalResponseType,
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

  SingleChoiceResponse as SingleChoiceResponseType,
  SingleFreeTextResponse as SingleFreeTextResponseType,
  SingleNumericalResponse as SingleNumericalResponseType,

229-246: ⚠️ Potential issue

Fix inconsistent field naming in FreeTextInstanceEvaluation

The implementation of SingleFreeTextResponse is good, but there's an inconsistency in the field naming:

  • NumericalInstanceEvaluation uses responses
  • FreeTextInstanceEvaluation uses answers

For consistency across evaluation types, we should use responses.

-      answers: t.expose('answers', {
+      responses: t.expose('responses', {
         type: [SingleFreeTextResponse],
         nullable: true,
       }),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

export const SingleFreeTextResponse = builder
  .objectRef<SingleFreeTextResponseType>('SingleFreeTextResponse')
  .implement({
    fields: (t) => ({
      value: t.exposeString('value'),
      count: t.exposeInt('count'),
    }),
  })

export const FreeTextInstanceEvaluation = builder
  .objectRef<IInstanceEvaluationFreeText>('FreeTextInstanceEvaluation')
  .implement({
    fields: (t) => ({
      ...sharedEvaluationProps(t),
      responses: t.expose('responses', {
        type: [SingleFreeTextResponse],
        nullable: true,
      }),

200-217: 💡 Codebase verification

⚠️ Potential issue

Fix typo in type name and update all references

The verification shows that the type with the typo SingleNumericalRepsonse is used in multiple places:

  • Type import alias
  • Object reference definition
  • Field type reference

The type is used in GraphQL operations through the NumericalInstanceEvaluation responses field, which is correctly spelled. The fix needs to be applied consistently across all occurrences.

  • packages/graphql/src/schema/question.ts:
    - SingleNumericalResponse as SingleNumericalRepsonseType,
    + SingleNumericalResponse as SingleNumericalResponseType,
    
    - export const SingleNumericalRepsonse = builder
    + export const SingleNumericalResponse = builder
    -   .objectRef<SingleNumericalRepsonseType>('SingleNumericalResponse')
    +   .objectRef<SingleNumericalResponseType>('SingleNumericalResponse')
    
    -         type: [SingleNumericalRepsonse],
    +         type: [SingleNumericalResponse],
🔗 Analysis chain

Fix typo in type name and verify response type usage

  1. The type name has a typo: SingleNumericalRepsonse should be SingleNumericalResponse
  2. The implementation looks good, but ensure all queries and mutations using this type are updated accordingly.
-export const SingleNumericalRepsonse = builder
+export const SingleNumericalResponse = builder
-  .objectRef<SingleNumericalRepsonseType>('SingleNumericalResponse')
+  .objectRef<SingleNumericalResponseType>('SingleNumericalResponse')
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Verify the usage of NumericalInstanceEvaluation responses field
rg -l "NumericalInstanceEvaluation.*responses" --type graphql

Length of output: 11982


Script:

#!/bin/bash
# Search for SingleNumericalRepsonse (with typo) usage
rg "SingleNumericalRepsonse" --type ts

# Search for NumericalInstanceEvaluation responses field usage
rg "NumericalInstanceEvaluation.*responses" --type ts -A 2

# Search for any GraphQL fragments or operations using NumericalInstanceEvaluation
fd -e graphql -X cat {} | rg "NumericalInstanceEvaluation"

Length of output: 59143

packages/types/src/index.ts (2)

465-470: 🛠️ Refactor suggestion

Rename answers to responses for uniformity

The IInstanceEvaluationFreeText interface uses the answers property, whereas similar data structures use responses in other interfaces. To maintain uniformity, consider renaming answers to responses.

Apply this diff to rename answers to responses:

 export type SingleFreeTextResponse = { count: number; value: string }
 export interface IInstanceEvaluationFreeText extends IBaseInstanceEvaluation {
-  answers?: SingleFreeTextResponse[]
+  responses?: SingleFreeTextResponse[]
   solutions: string[]
   lastResponse?: SingleQuestionResponseValue | null
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

export type SingleFreeTextResponse = { count: number; value: string }
export interface IInstanceEvaluationFreeText extends IBaseInstanceEvaluation {
  responses?: SingleFreeTextResponse[]
  solutions: string[]
  lastResponse?: SingleQuestionResponseValue | null
}

450-454: 🛠️ Refactor suggestion

Consider renaming choices to responses for consistency

In the IInstanceEvaluationChoices interface, the property is named choices, whereas in IInstanceEvaluationNumerical and IInstanceEvaluationFreeText, the properties are responses. For consistency across evaluation interfaces, consider renaming choices to responses.

Apply this diff to rename choices to responses:

 export type SingleChoiceResponse = { ix: number; count: number }
 export interface IInstanceEvaluationChoices extends IBaseInstanceEvaluation {
-  choices: SingleChoiceResponse[]
+  responses: SingleChoiceResponse[]
   lastResponse?: SingleQuestionResponseChoices | null
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

export type SingleChoiceResponse = { ix: number; count: number }
export interface IInstanceEvaluationChoices extends IBaseInstanceEvaluation {
  responses: SingleChoiceResponse[]
  lastResponse?: SingleQuestionResponseChoices | null
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant