-
Notifications
You must be signed in to change notification settings - Fork 0
Work Plan
Earlier ideas for this project fell into two categories of interest: (1) a gamified experience of the Library's Free to Use and Reuse Collection and (2) a collage tool that would allow for artistic exploration of the Library's Free to Use and Reuse Collection.
This project will combine both of these ideas, in which there will be a "flat" gamified narrative and activity component that the user will have to play/walkthrough to get to the collage aspect of the game. In other words, the user's end goal is the collage.
Here is an example of the sort of "flat" gaming experience we are thinking about.
Here is a brief scenario to demonstrate:
Screen prompt: On a dark and stormy night, {supervillain we create} enters the museum and swipes all the hats from the paintings! ...Later in the game... Screen prompt: Someone stole Jackie Robinson's hat! Complete the following puzzle to help him get it back.
...Later in the game... Screen prompt: Congratulations! You've found Jackie Robinson's hat!
Screen Prompt: You've unlocked Front cover of Jackie Robinson comic book. You can now wear Jackie Robinson's hat.
...Later in the game... Using image segmentation, user can dress their avatar in Jackie Robinson's hat.
End of demo.
The goal is to have students learn about a given image's historical framing. The collage aspect provides an end goal that entices participation, while the gamified/puzzle aspect ensures interaction with the material in a meaningful way.
The project will be broken into three main phases.
Phase 1: Choosing a collection.
Phase 2: There are two somewhat simultaneous working parts here. While building the image segmentation and avatar aspect of the deliverable, we will brainstorm the narrative surrounding the game.
Phase 3: Coding for the "connective" parts of the game -- the narrative and puzzles.
Age group: Later elementary students (4th and 5th grade). There is some "leakage" here into early middle school.
Use case: Civics/history. We envision that this can be part of a specific civiv/history lesson if a teacher wishes. The more likely scenario is an "educational play." (In elementary school, if you finished your work early or are in after school care, we'd be able to play educational games on the computers.)
Note: We are still exploring the exact set of images we'll use from the Free to Use and Reuse Collection.
Image Instance Segmentation
- labelme, Image Annotator tool: In many ways, I envision the collage tool working like the AI powered annotation features of this tool; enabling individuals to select and participate in the masking process before an item in the collage is cut-out: GNU General Public License
- EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything: This is the AI Model used in the labelme Annotator tool (labeled: EfficientSAM (Speed) that Aisaiah sampled with the NY content from the Free-to-Use. It has been reliable and quick so far in annotating; Apache-2.0 license
- Mask_RCNN.ipynb: A notebook from a tutorial on background remova. This is an example of masking and background tutorial.
- COCO Dataset: This is a small Computer Vision training dataset containing 80 categories of common objects in context. Should we elect to train a Computer Vision model this could be combined with formatted dataset as a basis.Creative Commons Attribution 4.0 License
- Interface hosted by the LoC or github pages that gives individuals access to an Library Collage Tool and an Interactive Game that leverages attributes of the tool.
List the major milestones, and a brief description and due date for each. Note that major deliverables should be complete by around week 8 (~July 12) to ensure enough time for Display Day prep and offboarding activities