Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to bulk generate anki cards #895

Merged
merged 60 commits into from
May 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
92c4f12
Add option to bulk generate anki cards
Kuuuube May 8, 2024
4e46589
Fix tab replacement
Kuuuube May 8, 2024
c47865c
Set deckname and modelname in note builder
Kuuuube May 9, 2024
cbf23a9
Add addNotes to ankiconnect api implementation
Kuuuube May 9, 2024
e35c33a
Add option to send word list to anki directly
Kuuuube May 9, 2024
0eb52d1
Add support for audio and media toggle
Kuuuube May 9, 2024
55074fe
Add support for dictionary media
Kuuuube May 9, 2024
edce401
Remove unnecessary assignment
Kuuuube May 9, 2024
967fb6d
Remove unused css
Kuuuube May 9, 2024
6728fba
Remove redundant html
Kuuuube May 9, 2024
102d8f6
Start of progress bar implementation
Kuuuube May 9, 2024
3e4f55a
Remove redundant type annotation
Kuuuube May 9, 2024
95a4d80
Remove unused import
Kuuuube May 9, 2024
4e2a6c9
Rename words to terms
Kuuuube May 9, 2024
8e17c4d
Print progress to console
Kuuuube May 9, 2024
9f24395
Add confirmation to Export to file
Kuuuube May 9, 2024
c7a2c88
Improve progress logs
Kuuuube May 9, 2024
df98c76
Add unresponsive and console note
Kuuuube May 9, 2024
414fc53
Add progress bars
Kuuuube May 9, 2024
fea1b6d
Make cancel button actually cancel operation
Kuuuube May 9, 2024
6a3a04b
Remove unresponsive warnings
Kuuuube May 9, 2024
dd59fb8
Disable send and export buttons after they are clicked
Kuuuube May 9, 2024
14a5172
Merge branch 'master' into bulk-card-creation
Kuuuube May 9, 2024
5d92b1d
Remove unneeded Yomichan mention
Kuuuube May 9, 2024
e39e646
Mark as experimental
Kuuuube May 9, 2024
41f49b1
Clarify description
Kuuuube May 9, 2024
7d79253
Add documentation on Anki Deck Generation
Kuuuube May 10, 2024
b8646ea
Add experimental note in docs
Kuuuube May 10, 2024
b13076b
Add warning text to settings page
Kuuuube May 10, 2024
6c635b9
Switch example text based on language
Kuuuube May 10, 2024
b18d3e5
Remove silly cancel function and bind directly
Kuuuube May 10, 2024
b4025e7
Rename to model
Kuuuube May 10, 2024
30a322e
Add link to docs
Kuuuube May 10, 2024
04fde9a
Make test text less confusing
Kuuuube May 10, 2024
cd8b726
Rename deck to notes
Kuuuube May 10, 2024
daa8efa
Clarify what is being sent to anki
Kuuuube May 10, 2024
68c68f1
Fix incorrect modal header text
Kuuuube May 10, 2024
a0b6d90
Clarify wording
Kuuuube May 10, 2024
b658a23
Fix ankiconnect addNotes return types
Kuuuube May 10, 2024
7eaf16b
Add error handling to send to anki
Kuuuube May 10, 2024
ccf14f8
Fix wording and naming in docs
Kuuuube May 10, 2024
31c53a7
Add option to prevent sending duplicates to anki
Kuuuube May 10, 2024
120bcff
Update anki deck and model without a page refresh
Kuuuube May 10, 2024
9c7d62e
Cleanup internal html naming
Kuuuube May 10, 2024
babc9a8
Cleanup type definition styling
Kuuuube May 10, 2024
5b464d9
Update example text without a page refresh
Kuuuube May 10, 2024
34bd779
Prevent closing the send/export confirm modal from messing up the ui …
Kuuuube May 10, 2024
a118ea7
Fix cancel getting stuck on true
Kuuuube May 10, 2024
bb447d0
Merge branch 'master' into bulk-card-creation
Kuuuube May 11, 2024
84033dd
Consolidate state changes
Kuuuube May 11, 2024
03d5cc6
Support idle download timeout
Kuuuube May 11, 2024
4eb83ff
Capitalize Failed to add cards error
Kuuuube May 11, 2024
2e7f5ea
Add separate variable for idleTimeout calculation
Kuuuube May 11, 2024
6d9f65a
Remove redundant _cachedDictionaryEntryValue variable
Kuuuube May 11, 2024
091288d
Use tags option to populate tags
Kuuuube May 11, 2024
843550b
Include deck and tags when exporting to file
Kuuuube May 11, 2024
baa0c98
Use date down to seconds and zero pad
Kuuuube May 11, 2024
894dfc8
Remove unnecessary ternary
Kuuuube May 11, 2024
5010ece
Limit 'path' finding function to only being able to search for 'path'
Kuuuube May 12, 2024
aadb1f4
Rename _findPathsByKey to _findAllPaths
Kuuuube May 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions ext/css/settings.css
Original file line number Diff line number Diff line change
Expand Up @@ -1640,6 +1640,43 @@ code.anki-field-marker {
min-height: calc(var(--textarea-line-height) * 5 + var(--textarea-padding) * 2);
}

.generate-anki-deck-layout {
display: flex;
flex-flow: column nowrap;
}
.generate-anki-deck-info {
flex: 0 1 auto;
}
.generate-anki-deck-test-input-container {
width: 100%;
}
.generate-anki-deck-test-container {
flex: 0 1 auto;
}
.generate-anki-deck-test-table {
display: grid;
grid-template-columns: 1fr auto;
grid-template-rows: auto;
align-items: center;
width: 100%;
box-sizing: border-box;
column-gap: 0.85em;
}
.generate-anki-deck-test-table-header {
font-size: var(--font-size-small);
}
#generate-anki-deck-textarea {
flex: 1 1 auto;
width: 100%;
max-width: 100%;
box-sizing: border-box;
resize: none;
min-height: calc(var(--textarea-line-height) * 5 + var(--textarea-padding) * 2);
}
#generate-anki-deck-test-text-input {
width: 100%;
}

.code {
flex: 0 0 auto;
width: 100%;
Expand Down
283 changes: 283 additions & 0 deletions ext/js/pages/settings/anki-deck-generator-controller.js
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,283 @@
/*
* Copyright (C) 2023-2024 Yomitan Authors
* Copyright (C) 2019-2022 Yomichan Authors
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/

import {ExtensionError} from '../../core/extension-error.js';
import {toError} from '../../core/to-error.js';
import {AnkiNoteBuilder} from '../../data/anki-note-builder.js';
import {getDynamicTemplates} from '../../data/anki-template-util.js';
import {querySelectorNotNull} from '../../dom/query-selector.js';
import {TemplateRendererProxy} from '../../templates/template-renderer-proxy.js';

export class AnkiDeckGeneratorController {
/**
* @param {import('./settings-controller.js').SettingsController} settingsController
* @param {import('./modal-controller.js').ModalController} modalController
* @param {import('./anki-controller.js').AnkiController} ankiController
*/
constructor(settingsController, modalController, ankiController) {
/** @type {import('./settings-controller.js').SettingsController} */
this._settingsController = settingsController;
/** @type {import('./modal-controller.js').ModalController} */
this._modalController = modalController;
/** @type {import('./anki-controller.js').AnkiController} */
this._ankiController = ankiController;
/** @type {?string} */
this._defaultFieldTemplates = null;
/** @type {HTMLTextAreaElement} */
this._wordInputTextarea = querySelectorNotNull(document, '#generate-anki-deck-textarea');
/** @type {HTMLInputElement} */
this._renderTextInput = querySelectorNotNull(document, '#generate-anki-deck-test-text-input');
/** @type {HTMLElement} */
this._renderResult = querySelectorNotNull(document, '#generate-anki-deck-render-result');
/** @type {HTMLElement} */
this._activeCardFormat = querySelectorNotNull(document, '#generate-anki-deck-active-card-format');
/** @type {string} */
this._notetype = '';
/** @type {?import('./modal.js').Modal} */
this._fieldTemplateResetModal = null;
/** @type {AnkiNoteBuilder} */
this._ankiNoteBuilder = new AnkiNoteBuilder(settingsController.application.api, new TemplateRendererProxy());
}

/** */
async prepare() {
this._defaultFieldTemplates = await this._settingsController.application.api.getDefaultAnkiFieldTemplates();

/** @type {HTMLButtonElement} */
const testRenderButton = querySelectorNotNull(document, '#generate-anki-deck-test-render-button');
/** @type {HTMLButtonElement} */
const generateButton = querySelectorNotNull(document, '#generate-anki-deck-generate-button');

testRenderButton.addEventListener('click', this._onRender.bind(this), false);
generateButton.addEventListener('click', this._onGenerate.bind(this), false);

void this._updateActiveCardFormat();
}

// Private

/**
*
*/
async _updateActiveCardFormat() {
const activeCardFormat = /** @type {HTMLElement} */ (this._activeCardFormat);
const options = await this._settingsController.getOptions();
this.notetype = options.anki.terms.model;
activeCardFormat.textContent = this.notetype;
}

/**
* @param {MouseEvent} e
*/
async _onGenerate(e) {
e.preventDefault();
const words = /** @type {HTMLTextAreaElement} */ (this._wordInputTextarea).value.split('\n');
let ankiTSV = '#separator:tab\n#html:true\n#notetype column:1\n';
for (const value of words) {
if (!value) { continue; }
const noteData = await this._generateNoteData(value, 'term-kanji');
const fieldsTSV = noteData ? this._fieldsToTSV(noteData) : '';
if (fieldsTSV) {
ankiTSV += this.notetype + '\t';
ankiTSV += fieldsTSV;
ankiTSV += '\n';
}
}
const today = new Date();
const fileName = `anki-deck-${today.getFullYear()}-${today.getMonth()}-${today.getDay()}.txt`;
const blob = new Blob([ankiTSV], {type: 'application/octet-stream'});
this._saveBlob(blob, fileName);
}

/**
* @param {HTMLElement} infoNode
* @param {import('anki-templates-internal').CreateModeNoTest} mode
* @param {boolean} showSuccessResult
*/
async _testNoteData(infoNode, mode, showSuccessResult) {
/** @type {Error[]} */
const allErrors = [];
const text = /** @type {HTMLInputElement} */ (this._renderTextInput).value;
let result;
try {
const noteData = await this._generateNoteData(text, mode);
result = noteData ? this._fieldsToTSV(noteData) : `No definition found for ${text}`;
} catch (e) {
allErrors.push(toError(e));
}

/**
* @param {Error} e
* @returns {string}
*/
const errorToMessageString = (e) => {
if (e instanceof ExtensionError) {
const v = e.data;
if (typeof v === 'object' && v !== null) {
const v2 = /** @type {import('core').UnknownObject} */ (v).error;
if (v2 instanceof Error) {
return v2.message;
}
}
}
return e.message;
};

const hasError = allErrors.length > 0;
infoNode.hidden = !(showSuccessResult || hasError);
if (hasError || !result) {
infoNode.textContent = allErrors.map(errorToMessageString).join('\n');
} else {
infoNode.textContent = showSuccessResult ? result : '';
}
infoNode.classList.toggle('text-danger', hasError);
}

/**
* @param {string} word
* @param {import('anki-templates-internal').CreateModeNoTest} mode
* @returns {Promise<?import('anki.js').NoteFields>}
*/
async _generateNoteData(word, mode) {
const optionsContext = this._settingsController.getOptionsContext();
const data = await this._getDictionaryEntry(word, optionsContext);
if (data === null) {
return null;
}
const {dictionaryEntry, text: sentenceText} = data;
const options = await this._settingsController.getOptions();
const context = {
url: window.location.href,
sentence: {
text: sentenceText,
offset: 0
},
documentTitle: document.title,
query: sentenceText,
fullQuery: sentenceText
};
const template = this._getAnkiTemplate(options);
const deckOptionsFields = options.anki.terms.fields;
const {general: {resultOutputMode, glossaryLayoutMode, compactTags}} = options;
const fields = [];
for (const deckField in deckOptionsFields) {
if (Object.prototype.hasOwnProperty.call(deckOptionsFields, deckField)) {
fields.push([deckField, deckOptionsFields[deckField]]);
}
}
const {note} = await this._ankiNoteBuilder.createNote(/** @type {import('anki-note-builder').CreateNoteDetails} */ ({
dictionaryEntry,
mode,
context,
template,
deckName: '',
modelName: '',
fields: fields,
resultOutputMode,
glossaryLayoutMode,
compactTags
}));
return note.fields;
}

/**
* @param {string} text
* @param {import('settings').OptionsContext} optionsContext
* @returns {Promise<?{dictionaryEntry: import('dictionary').TermDictionaryEntry, text: string}>}
*/
async _getDictionaryEntry(text, optionsContext) {
const {dictionaryEntries} = await this._settingsController.application.api.termsFind(text, {}, optionsContext);
if (dictionaryEntries.length === 0) { return null; }

this._cachedDictionaryEntryValue = dictionaryEntries[0];
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
return {
dictionaryEntry: /** @type {import('dictionary').TermDictionaryEntry} */ (this._cachedDictionaryEntryValue),
text: text
};
}

/**
* @param {import('settings').ProfileOptions} options
* @returns {string}
*/
_getAnkiTemplate(options) {
let staticTemplates = options.anki.fieldTemplates;
if (typeof staticTemplates !== 'string') { staticTemplates = this._defaultFieldTemplates; }
const dynamicTemplates = getDynamicTemplates(options);
return staticTemplates + '\n' + dynamicTemplates;
}

/**
* @param {Event} e
*/
_onRender(e) {
e.preventDefault();

const infoNode = /** @type {HTMLElement} */ (this._renderResult);
infoNode.hidden = true;
void this._testNoteData(infoNode, 'term-kanji', true);
}

/**
* @param {import('anki.js').NoteFields} noteFields
* @returns {string}
*/
_fieldsToTSV(noteFields) {
let tsv = '';
for (const key in noteFields) {
if (Object.prototype.hasOwnProperty.call(noteFields, key)) {
tsv += noteFields[key].replace('\t', '&nbsp;&nbsp;&nbsp;') + '\t';
Fixed Show fixed Hide fixed
}
}
return tsv;
}

/**
* @param {Blob} blob
* @param {string} fileName
*/
_saveBlob(blob, fileName) {
if (
typeof navigator === 'object' && navigator !== null &&
// @ts-expect-error - call for legacy Edge
typeof navigator.msSaveBlob === 'function' &&
// @ts-expect-error - call for legacy Edge
navigator.msSaveBlob(blob)
) {
return;
}

const blobUrl = URL.createObjectURL(blob);

const a = document.createElement('a');
a.href = blobUrl;
a.download = fileName;
a.rel = 'noopener';
a.target = '_blank';

const revoke = () => {
URL.revokeObjectURL(blobUrl);
a.href = '';
this._settingsExportRevoke = null;
};
this._settingsExportRevoke = revoke;

a.dispatchEvent(new MouseEvent('click'));
setTimeout(revoke, 60000);
}
}
4 changes: 4 additions & 0 deletions ext/js/pages/settings/settings-main.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {DocumentFocusController} from '../../dom/document-focus-controller.js';
import {querySelectorNotNull} from '../../dom/query-selector.js';
import {ExtensionContentController} from '../common/extension-content-controller.js';
import {AnkiController} from './anki-controller.js';
import {AnkiDeckGeneratorController} from './anki-deck-generator-controller.js';
import {AnkiTemplatesController} from './anki-templates-controller.js';
import {AudioController} from './audio-controller.js';
import {BackupController} from './backup-controller.js';
Expand Down Expand Up @@ -117,6 +118,9 @@ await Application.main(true, async (application) => {
const ankiController = new AnkiController(settingsController);
preparePromises.push(ankiController.prepare());

const ankiDeckGeneratorController = new AnkiDeckGeneratorController(settingsController, modalController, ankiController);
preparePromises.push(ankiDeckGeneratorController.prepare());

const ankiTemplatesController = new AnkiTemplatesController(settingsController, modalController, ankiController);
preparePromises.push(ankiTemplatesController.prepare());

Expand Down
37 changes: 37 additions & 0 deletions ext/settings.html
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -1842,6 +1842,14 @@ <h1>Yomitan Settings</h1>
<button type="button" class="icon-button"><span class="icon-button-inner"><span class="icon" data-icon="material-right-arrow"></span></span></button>
</div>
</div></div>
<div class="settings-item settings-item-button advanced-only" data-modal-action="show,generate-anki-deck"><div class="settings-item-inner">
<div class="settings-item-left">
<div class="settings-item-label">Generate Anki Deck&hellip;</div>
</div>
<div class="settings-item-right open-panel-button-container">
<button type="button" class="icon-button"><span class="icon-button-inner"><span class="icon" data-icon="material-right-arrow"></span></span></button>
</div>
</div></div>
</div>

<!-- Clipboard -->
Expand Down Expand Up @@ -3213,6 +3221,35 @@ <h1>Yomitan Settings</h1>
</div>
</div></div>

<!-- Generate anki deck modal -->
<div id="generate-anki-deck-modal" class="modal" tabindex="-1" role="dialog" hidden><div class="modal-content modal-content-full">
<div class="modal-header"><div class="modal-title">Anki Deck Generator</div></div>
<div class="modal-body generate-anki-deck-layout">
<div class="generate-anki-deck-info">
<p>
Enter a newline separated list of words below to generate an Anki deck in <code>Notes in plain text (.txt)</code> format.
</p>
</div>
<textarea autocomplete="off" spellcheck="false" id="generate-anki-deck-textarea" class="no-wrap margin-above" data-tab-action="indent,4"></textarea>
<div class="generate-anki-deck-test-container margin-above">
<p>
Active Anki card format: <code id="generate-anki-deck-active-card-format"></code>
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
</p>
<div class="generate-anki-deck-test-table margin-above">
<div class="generate-anki-deck-test-table-header">Test word</div>
<div></div>
<input type="text" id="generate-anki-deck-test-text-input" class="form-control" value="読め" placeholder="Preview text" autocomplete="off" lang="ja">
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
<button type="button" id="generate-anki-deck-test-render-button">Test</button>
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
</div>
</div>
<div class="code margin-above" id="generate-anki-deck-render-result"><em>Card render result</em></div>
Kuuuube marked this conversation as resolved.
Show resolved Hide resolved
</div>
<div class="modal-footer">
<button type="button" class="low-emphasis" data-modal-action="generate" id="generate-anki-deck-generate-button">Generate</button>
<button type="button" data-modal-action="hide">Close</button>
</div>
</div></div>


<!-- Import/export modals -->
<div id="settings-import-error-modal" class="modal" tabindex="-1" role="dialog" hidden><div class="modal-content modal-content-small">
Expand Down