Skip to content

[0.31.1] Update for AE 1.6.1170

Latest
Latest
Compare
Choose a tag to compare
@SwimmingTiger SwimmingTiger released this 09 Mar 09:09
· 2 commits to master since this release
v0.31.1-alpha
02def5b

To run DSN, the game version must be:

  • Skyrim Special Edition / Anniversary Edition 1.6.1170 (with or without DLC), supported since 0.31.1.
    Only tested with the Steam version, not sure if the GOG version will work.
  • Skyrim Special Edition 1.5.97
  • Skyrim VR 1.4.15

Update

[0.31.1] Compatible with version 1.6.1170 of SE/AE.

About the New Engine

A new optional speech recognition engine Voice2Json has been added to DSN since v0.30.0.

Voice2Json Available Locales

ca-es, cs-cz, de, el-gr, en, en-in, en-us, es, es-mexican,
fr, hi, it, ko-kr, kz, nl, pl, pt-br, ru, sv, vi, zh, zh-cn

Voice2Json MAY recognize the following languages better:

cs-cz, de, en, en-us, es, fr, it, ko-kr, nl, sv, vi

Microsoft speech recognition engine is recommended for other languages.

Official language support page: http://voice2json.org/#supported-languages

If you see the word kaldi after a language, it should be recognized better. If the words behind it are pocketsphinx, the recognition performance is poor.

How about deepspeech? I think it's somewhere between kaldi and pocketsphinx. You can test it yourself.

Special note: Voice2Json's Chinese support is currently poor, and it is not recommended for Chinese users. The Chinese language support of Microsoft's speech recognition engine is much better. (synesthesiam/voice2json#76)

[zhCN] 特别说明：Voice2Json的汉语支持目前很烂，不建议汉语用户使用。微软语音识别引擎的汉语识别效果好得多。(synesthesiam/voice2json#76)

Voice2Json speech engine is optional

The follow is how to install the Voice2Json engine for DSN, but the engine is always optional. If you remove the Engine=Voice2Json line from the configuration file, the Microsoft Speech Recognition Engine will be used. Then you can use the alpha version like the beta version, and you don't need to install Docker for Windows.

Install DSN with Voice2Json

  1. Install xSHADOWMANx's Dll Loader manully (without mod manager) to the Skyrim root directory (the directory that has SkyrimSE.exe or SkyrimVR.exe). It is a mandatory dependency of DSN.

  2. Install Docker Desktop in your system: https://www.docker.com/products/docker-desktop/
    If available, check "Use WSL2 instead of Hyper-V" when installing.
    Running Docker Desktop in WSL2 will provide better performance and flexibility than Hyper-V.

  3. If your Docker Desktop doesn't launch properly, try running the following commands with Admin PowerShell:

    wsl --update
wsl --shutdown

    Then restart your Docker Desktop.

    If still unsuccessful, you need to enable CPU virtualization (Intel VT-x or AMD-V) in the BIOS settings.

    If docker still doesn't work for you, you can try to install Voice2Json in WSL1 and uncomment bVoice2JsonUseDocker=0 in DragonbornSpeaksNaturally.ini. There is currently no comprehensive guide, but you can refer to this comment.

    More about WSL1 and WSL2: https://aka.ms/wsl2

  4. Download the DragonbornSpeaksNaturally-Voice2Json-Alpha-0.31.x.zip file from Assets at below and install with your mod manager.

  5. Rename DragonbornSpeaksNaturally.SAMPLE.ini to DragonbornSpeaksNaturally.ini and
    open it with your text editor, change the Locale=en_us to the value you want. It could be:

    ca-es, cs-cz, de, el-gr, en, en-in, en-us, es, es-mexican,
    fr, hi, it, ko-kr, kz, nl, pl, pt-br, ru, sv, vi, zh, zh-cn

    If you want to try a different backend recognizer, you can fill in the Locale containing the recognizer, for example:

    Locale=en-us_deepspeech-mozilla

    See the recognizer backends available for each language.

    If you want to use the old configuration file, edit the following options:

    ;;; Please do not edit, delete or move the next line (the ini section title), otherwise the options below will not take effect
[SpeechRecognition]

;;; Uncomment (remove leading semicolon) to choose speech engine (default is Microsoft)
;;;
;;; Voice2Json recognizes the following languages better:
;;;     cs-cz, de, en, en-us, es, fr, it, ko-kr, nl, ru, sv, vi
;;; Microsoft is recommended for other languages.
;;;
;;; To use the Voice2Json engine, you need to install Docker Desktop:
;;;     Download from https://www.docker.com/products/docker-desktop/
;;;
;Engine=Microsoft
;;; 【You need to add this】
Engine=Voice2Json

;;; Set this to override your system's default locale
;;;
;;; Available Locales for Engine=Microsoft:
;;;     en-us, en-uk, en-ca, en-in, en-au, fr, de, ja,
;;;     zh-cn, zh-tw, zh-hk, zh-sg
;;;
;;; Available Locales for Engine=Voice2Json:
;;;     ca-es, cs-cz, de, el-gr, en, en-in, en-us, es, es-mexican, 
;;;     fr, hi, it, ko-kr, kz, nl, pl, pt-br, ru, sv, vi, zh, zh-cn
;;; References: http://voice2json.org/#supported-languages
;;;
;;; 【You need to change this】
Locale=en-us

;;; When set to 1, the speech recognition service will log any audio signal issues like "too loud" or "too noisy"
;;; 【Enable this for debug】
bLogAudioSignalIssues=1

  6. Launch the game and DSN will do the rest automatically.
    See My Documents\DragonbornSpeaksNaturally\DragonbornSpeaksNaturally.log for the launching progress.

  7. If you don't launch Docker Desktop, DSN will try to launch it for you and a UAC popup will appear. (Why we need UAC?)
    If you don't want the UAC popup, please launch Docker Desktop manually before launching Skyrim.

  8. On first launch, an container image of several hundred megabytes will be downloaded from Docker Hub.
    So the speech recognition may not ready anytime soon at the first time.
    See My Documents\DragonbornSpeaksNaturally\DragonbornSpeaksNaturally.log for the downloading progress.

  9. Every time you change the Locale, Voice2Json needs to download some speech recognition model files from github.
    Once the model is downloaded, all other operations do not require an Internet connection.
    Speech recognition is done entirely locally and your voice is not uploaded to any server.

  10. If for some reason DSN cannot create the docker container automatically, you can create it manually:

    docker run -dit --name dsn_voice2json --entrypoint /bin/sh synesthesiam/voice2json

    And download the model files:

    docker exec dsn_voice2json /usr/lib/voice2json/bin/voice2json -p en-us download-profile

    Train the model files (with the default texts, just for test):

    docker exec dsn_voice2json /usr/lib/voice2json/bin/voice2json -p en-us train-profile

    Change en-us to your locale.

  11. If your dsn_voice2json container is corrupted for some reason, you can remove it with the command:

    docker rm -f dsn_voice2json

    You can also remove it from the Docker Desktop GUI.
    The next time you launch DSN, it will automatically be recreated. You can also recreate it manually with the step 10.

Known Issues

  1. SRGS.xml does not work with the Voice2Json engine, so the Dragonborn Unlimited configuration file will not work.

  2. SubsetMatchingMode option doesn't work, but optionalExpression works fine. So you won't be able to omit words in a sentence unless they are in parentheses.

  3. When launching for the first time in a new language, the dsn_voice2json docker container will download some model files for the language from Github. If you have a poor connection to GitHub and the model cannot be downloaded, Voice2Json will not work properly. If this occurred, do step 10 to manually download the model files, or get a VPN for your network connection temporarily (Once step 10 is done, the VPN or network connection is no longer needed).

FAQ

Q: Do I need to download voice2jason? and if so where and how. Description is unclear.

A: When you launch Skyrim for the first time after installing Docker for Windows, DSN will automatically install voice2json with the docker command for you. When the installation is complete, you will see a dsn_voice2json container in the Docker Desktop UI.

image

Q: What's inside the Data\Plugins\Sumwunn\Resources\ folder?

A: These are the files to support Voice2Json Chinese speech recognition. There is no space between words in Chinese sentences, while Voice2Json can only accept input in word units. This makes DSN have to complete Chinese word segmentation by itself before using Voice2Json for Chinese speech recognition. And the Resources folder contains the dictionary files needed to implement it.

If you don't use Voice2Json for Chinese speech recognition, you can delete this folder.

Q: Some special symbols in the sentence make the recognizer error, how can I get rid of them?

A: In DragonbornSpeaksNaturally.ini, for a single character:

; ...
[SpeechRecognition]
; ...
; Remove the semicolon `;` in front of the following two lines and
; add the special symbol you want to remove after the `,`
normalizeExpression=(?:[",!?]|\s+|(?<![a-zA-Z])')
normalizeReplacement=" "

For multiple characters:

normalizeExpression=(?:[",!?]|\s+|(?<![a-zA-Z])'|what you want to remove|another content you want to remove)
Assets 3
Loading