diff --git a/Project_Doc/Test_Report.md b/Project_Doc/Test_Report.md index 81b878d5..b98fd4d2 100644 --- a/Project_Doc/Test_Report.md +++ b/Project_Doc/Test_Report.md @@ -1,64 +1,46 @@ # CookHub -
- --> Which tools --> Statistic required about: How many tests passed, how many failed ## Test Report -
### 1. Introduction > This section of the documentation provides an overview of the software testing process adopted in our project, focusing specifically on the utilization of GitHub Actions and unit testing with TypeScript and Selenium. -**Overview of Testing Strategy** -> We implement integrated testing, automated workflows via GitHub Actions and unit testing in TypeScript. - -**GitHub Actions** -> GitHub Actions play a pivotal role in automating our testing pipelines. Every code commit triggers a series of actions that execute automated tests, ensuring that all changes are validated against predefined test cases before integration. +// TODO -**Selenium** -> The Selenium WebDriver drives a browser natively, as a user would, either locally or on a remote machine using the Selenium server, marks a leap forward in terms of browser automation. -> Selenium WebDriver refers to both the language bindings and the implementations of the individual browser controlling code. -> Also see [Selenium](https://github.com/SE-TINF22B6/CookHub/wiki/Testing) in our Wiki. - -**Unit Testing with TypeScript** -> Unit tests form the backbone of our testing framework, primarily focusing on the functionality of the smallest parts of the program. Written in TypeScript, these tests verifying the logic and behavior of individual modules. +### 2. Test Strategy +> This section outlines our overall approach to testing within the project. It covers the methodologies, types of tests, and specific techniques we employ, alongside the rationale behind our selective testing strategy. -**Scope of Testing Activities** -> The scope of our testing extends from simple code functionality checks to complex interaction and integration tests, ensuring that (nearly) every component works as expected. +#### Used technoligies: -**Expected Outcomes** -> By the end of this section, readers should have a thorough understanding of the testing processes, equipped with the knowledge to execute and improve upon the existing tests. +*GitHub Actions* +- GitHub Actions play a pivotal role in automating our testing pipelines. Every code commit triggers a series of actions that execute automated tests, ensuring that all changes are validated against predefined test cases before integration. -

+*Jest* +- To ensure comprehensive testing and maintain high code quality, we have decided to integrate Jest as our primary testing framework fort the Frontend. By utilizing Jest's robust features, such as snapshot testing and mock functions, we aim to streamline our testing process. We also agreed on setting up individual test suites for each module, allowing us to isolate issues quickly and maintain a clear structure. Regular test execution will be enforced to catch bugs early and ensure our code remains reliable and maintainable. -### 2. Test Strategy -> This section outlines our overall approach to testing within the project. It covers the methodologies, types of tests, and specific techniques we employ, alongside the rationale behind our selective testing strategy due to limited time and resources. +*xUnit.net* +- To ensure thorough testing and maintain high code quality, we have decided to implement XUnit.net as our primary testing framework for the Backend. By leveraging XUnit.net's powerful features, such as parallel test execution and extensibility, we aim to enhance our testing efficiency. We also agreed on organizing individual test cases for each module, which will help us quickly identify and resolve issues while keeping our codebase structured. Regular test runs will be enforced to catch bugs early, ensuring our code remains reliable and maintainable. -**Testing Methodology** -> Our methodology is centered around agile testing practices, which are integrated into the development cycle. This approach allows for continuous testing and immediate feedback, crucial for addressing issues promptly and efficiently. Testing activities are automated as much as possible to streamline the process and minimize manual effort. +*Selenium* +- To ensure robust end-to-end testing and maintain high quality in our web applications, we have decided to adopt Selenium as our primary automation tool. By utilizing Selenium's capabilities for browser automation, we aim to simulate real user interactions and identify potential issues early in the development process. We also agreed on developing individual test scripts for each feature, allowing us to isolate and address issues promptly while maintaining a well-organized test suite. Regular execution of automated tests will be enforced to ensure our applications remain reliable, user-friendly, and free of critical bugs. -**Types of Testing** -> Given our resource constraints, we focus primarily on the following types of tests: -> -> - **Unit Testing:** The foundation of our testing efforts, focusing on individual units of code to ensure they function correctly in isolation. These tests are automated through frameworks tailored for TypeScript. -> - **Integration Testing:** Conducted to verify that different modules or services work together as expected. These tests are crucial when new features that depend on existing functionalities are developed. -> - **Regression Testing:** Automated to ensure that recent changes have not adversely affected existing functionalities. These tests are triggered automatically via GitHub Actions whenever changes are pushed to the repository. +#### Testing Methodology +Our methodology is centered around agile testing practices, which are integrated into the development cycle. This approach allows for continuous testing and immediate feedback, crucial for addressing issues promptly and efficiently. Testing activities are automated as much as possible to streamline the process and minimize manual effort. -**Testing Tools and Frameworks** -> The following tools and frameworks are integral to our testing strategy: +#### Types of Testing +Given our resource constraints, we focus primarily on the following types of tests: -> - **Jest:** Our primary framework for unit testing. It is well-suited for the TypeScript environment, offering robust testing capabilities, mock functions, and asynchronous testing support. -> - **GitHub Actions:** Used for automating our CI/CD pipeline, including running our test suites on every push and pull request. This helps in identifying and resolving issues early in the development cycle. -> - **ESLint:** While primarily a linting tool, ESLint helps ensure that our TypeScript code adheres to best practices and standards, indirectly supporting our testing efforts by reducing the likelihood of bugs. +- **Unit Testing:** The foundation of our testing efforts, focusing on individual units of code to ensure they function correctly in isolation. These tests are automated through frameworks tailored for TypeScript. +- **Integration Testing:** Conducted to verify that different modules or services work together as expected. These tests are crucial when new features that depend on existing functionalities are developed. +- **End-to-end Testing:** To make sure that the flow of data is maintained for all kinds of user tasks and processes. This type of testing approach starts from the end user's perspective and simulates a real-world scenario. **Scope Limitations** -> Due to the constraints of time and resources in our study project, comprehensive testing like full system testing or exhaustive end-to-end testing will not be prioritized. Our focus will remain on critical functionalities and integration points, aiming to maximize our testing impact within the available means. +Due to the constraints of time and resources in our study project, comprehensive testing like full system testing or exhaustive end-to-end testing will not be prioritized. Our focus will remain on critical functionalities and integration points, aiming to maximize our testing impact within the available means. **Expected Outcomes** -> The primary objective of our testing strategy is to maintain a high standard of quality for all delivered features, despite the restricted scope. By leveraging automated tools and focusing on key areas of the software, we aim to ensure stability and functionality, minimizing the risk of major defects in the production environment. -> -> This testing strategy is designed to be adaptive and scalable, able to accommodate more comprehensive testing phases as resources become available, ensuring ongoing improvement in the quality and reliability of the software product. +The primary objective of our testing strategy is to maintain a high standard of quality for all delivered features, despite the restricted scope. By leveraging automated tools and focusing on key areas of the software, we aim to ensure stability and functionality, minimizing the risk of major defects in the production environment. + +This testing strategy is designed to be adaptive and scalable, able to accommodate more comprehensive testing phases as resources become available, ensuring ongoing improvement in the quality and reliability of the software product.

@@ -111,6 +93,11 @@ out of 1,073 recorded workflow runs, there where: ### 7. Recommendations > This section offers suggestions for improving the testing process and the quality of the software. +We noticed some things that could be improved in the tools that we used: +- Jest + - Jest provides an easy way to write tests in a readable way, but sometimes lacks feedback when a test failed. We often had the problem that a test got stuck at some point, and Jests error message did not include the line of code. This made it hard to find out the reason of failure. +- Selenium + - Selenium is great once it works, but it took a long time to set it up. This is because there are many different requirements (having to run with admin privileges, no support for the newest node.js versions) and setting it up in the CI/CD pipeline also requires a lot of steps. You also have to be very careful to write tests in a way that they run reliably, and that there are no race conditions. This is a challenge because selenium only offers a limited number of ways to manually wait for certain things to load