Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VOICEVOX (AI speech synthesis) package for ROS Interface #337

Open
wants to merge 31 commits into
base: master
Choose a base branch
from

Conversation

iory
Copy link
Member

@iory iory commented Apr 17, 2022

voicevox

ROS Interface for VOICEVOX (AI speech synthesis)

TERM

VOICEVOX is basically free to use, but please check the terms of use below.

TERM

Each voice synthesis character has its own rules. Please use this package according to those terms.

Character name term link
四国めたん https://zunko.jp/con_ongen_kiyaku.html
ずんだもん https://zunko.jp/con_ongen_kiyaku.html
春日部つむぎ https://tsukushinyoki10.wixsite.com/ktsumugiofficial/利用規約
波音リツ http://canon-voice.com/kiyaku.html
雨晴はう https://amehau.com/?page_id=225
玄野武宏 https://virvoxproject.wixsite.com/official/voicevoxの利用規約
白上虎太郎 https://virvoxproject.wixsite.com/official/voicevoxの利用規約
青山龍星 https://virvoxproject.wixsite.com/official/voicevoxの利用規約
冥鳴ひまり https://kotoran8zunzun.wixsite.com/my-site/利用規約
九州そら https://zunko.jp/con_ongen_kiyaku.html

Installation

Build this package.

cd /path/to/catkin_workspace
catkin build voicevox

Usage

Launch sound_play with VOICEVOX Text-to-Speech

roslaunch voicevox voicevox_texttospeech.launch

Say something

For python users

import rospy
from sound_play.libsoundplay import SoundClient

rospy.init_node('say_node')

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp')

client.say('こんにちは', voice='四国めたん-あまあま')

You can change the voice by changing the voice_name.
You can also specify the speaker id.
Look at the following tables for further details.

speaker_id voice_name
0 四国めたん-あまあま
1 ずんだもん-あまあま
2 四国めたん-ノーマル
3 ずんだもん-ノーマル
4 四国めたん-セクシー
5 ずんだもん-セクシー
6 四国めたん-ツンツン
7 ずんだもん-ツンツン
8 春日部つむぎ-ノーマル
9 波音リツ-ノーマル
10 雨晴はう-ノーマル
11 玄野武宏-ノーマル
12 白上虎太郎-ノーマル
13 青山龍星-ノーマル
14 冥鳴ひまり-ノーマル
15 九州そら-あまあま
16 九州そら-ノーマル
17 九州そら-セクシー
18 九州そら-ツンツン
19 九州そら-ささやき

For roseus users

$ roseus
(load "package://pr2eus/speak.l")

(ros::roseus "say_node")

(speak "JSKへようこそ。" :lang "波音リツ" :wait t :topic-name "robotsound_jp")

Tips

Normally, the server for speech synthesis starts up at http://localhost:50021.
You can change the url and port by setting values for VOICEVOX_TEXTTOSPEECH_URL and VOICEVOX_TEXTTOSPEECH_PORT.

You can also set the default character by setting VOICEVOX_DEFAULT_SPEAKER_ID.
Please refer to here for the speaker id.

@tkmtnt7000
Copy link
Member

tkmtnt7000 commented Apr 18, 2022

That's a fascinating node!
I got a following error in melodic when executing catkin build voicevox.

I think this error is related to pypa/setuptools#1694, but I have not found the solution yet.....

tsukamoto@tsukamoto-desktop ~/tmp/voicevox_ws/src/jsk_3rdparty/3rdparty/voicevox (voicevox *%=) 
$ catkin build voicevox
--------------------------------------------------------------------
Profile:                     default
Extending:             [env] /opt/ros/melodic
Workspace:                   /home/tsukamoto/tmp/voicevox_ws
--------------------------------------------------------------------
Build Space:        [exists] /home/tsukamoto/tmp/voicevox_ws/build
Devel Space:        [exists] /home/tsukamoto/tmp/voicevox_ws/devel
Install Space:      [unused] /home/tsukamoto/tmp/voicevox_ws/install
Log Space:         [missing] /home/tsukamoto/tmp/voicevox_ws/logs
Source Space:       [exists] /home/tsukamoto/tmp/voicevox_ws/src
DESTDIR:            [unused] None
--------------------------------------------------------------------
Devel Space Layout:          linked
Install Space Layout:        None
--------------------------------------------------------------------
Additional CMake Args:       None
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
--------------------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
--------------------------------------------------------------------
Workspace configuration appears valid.

NOTE: Forcing CMake to run for each package.
--------------------------------------------------------------------
[build] Found '33' packages in 0.0 seconds.                                                  
[build] Updating package table.                                                              
Starting  >>> catkin_tools_prebuild                                                          
Finished  <<< catkin_tools_prebuild                [ 2.8 seconds ]                           
Starting  >>> voicevox                                                                       
_____________________________________________________________________________________________
Errors     << voicevox:make /home/tsukamoto/tmp/voicevox_ws/logs/voicevox/build.make.000.log 
make[3]: 警告: jobserver が利用不可: 今回は -j1 を使います. 親 make ルールに `+' を追加しましょう.
make[3]: 警告: jobserver が利用不可: 今回は -j1 を使います. 親 make ルールに `+' を追加しましょう.
    ERROR: Command errored out with exit status 1:
     command: /home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/bin/python /home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpdsh3vj7w
         cwd: /tmp/pip-resolver-jgzbc29r/pyopenjtalk
    Complete output (10 lines):
    Traceback (most recent call last):
      File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 108, in get_requires_for_build_wheel
        backend = _build_backend()
      File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 99, in _build_backend
        obj = getattr(obj, path_part)
    AttributeError: module 'setuptools.build_meta' has no attribute '__legacy__'
    ----------------------------------------
Traceback (most recent call last):
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/piptools/scripts/compile.py", line 392, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/piptools/resolver.py", line 169, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/piptools/resolver.py", line 274, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/piptools/resolver.py", line 363, in _iter_dependencies
    for dependency in self.repository.get_dependencies(ireq):
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 228, in get_dependencies
    download_dir, ireq, wheel_cache
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 180, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 431, in prepare_linked_requirement
    req, self.req_tracker, self.finder, self.build_isolation,
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 85, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/wrappers.py", line 258, in _call_hook
    extra_environ=extra_environ
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/utils/subprocess.py", line 274, in runner
    spinner=spinner,
  File "/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_internal/utils/subprocess.py", line 241, in call_subprocess
    raise InstallationError(exc_msg)
pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/bin/python /home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpdsh3vj7w Check the logs for full command output.
Traceback (most recent call last):
  File "/opt/ros/melodic/lib/catkin_virtualenv/venv_lock", line 51, in <module>
    extra_pip_args=[arg for arg in extra_pip_args.split(" ") if arg != ""],
  File "/opt/ros/melodic/lib/python2.7/dist-packages/catkin_virtualenv/venv.py", line 143, in lock
    run_command(command, check=True)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/catkin_virtualenv/__init__.py", line 44, in run_command
    return subprocess.run(cmd, *args, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/tsukamoto/tmp/voicevox_ws/build/voicevox/venv/bin/pip-compile', '--no-header', 'requirements.in', '--pip-args', '-qq --retries 10 --timeout 30', '-o', '/home/tsukamoto/tmp/voicevox_ws/src/jsk_3rdparty/3rdparty/voicevox/requirements.txt']' returned non-zero exit status 1.
make[2]: *** [/home/tsukamoto/tmp/voicevox_ws/src/jsk_3rdparty/3rdparty/voicevox/requirements.txt] Error 1
make[1]: *** [CMakeFiles/voicevox_generate_virtualenv.dir/all] Error 2
make[1]: *** 未完了のジョブを待っています....
make: *** [all] Error 2
cd /home/tsukamoto/tmp/voicevox_ws/build/voicevox; catkin build --get-env voicevox | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -
.............................................................................................
Failed     << voicevox:make                        [ Exited with code 2 ]                    
Failed    <<< voicevox                             [ 35.7 seconds ]                          
[build] Summary: 1 of 2 packages succeeded.                                                  
[build]   Ignored:   32 packages were skipped or are blacklisted.                            
[build]   Warnings:  None.                                                                   
[build]   Abandoned: None.                                                                   
[build]   Failed:    1 packages failed.                                                      
[build] Runtime: 38.6 seconds total.                                                         
[build] Note: Workspace packages have changed, please re-source setup files to use them.
```l

@iory
Copy link
Member Author

iory commented Apr 18, 2022

What is the results of

dpkg -l | grep python3-setuptools
pip3 freeze | grep setuptools

@tkmtnt7000
Copy link
Member

tkmtnt7000 commented Apr 18, 2022

$ dpkg -l | grep python3-setuptools
ii  python3-setuptools                              39.0.1-2                                        all          Python3 Distutils Enhancements
$ pip3 freeze | grep setuptools
;; no result

After confirming setuptools version, I installed setuptools==59.6.0 manually with executing pip3 install setuptools, but the error above still occured.

@iory
Copy link
Member Author

iory commented Apr 18, 2022

Sorry, I was wrong.
What is the results of

dpkg -l | grep python-setuptools
pip freeze | grep setuptools

@tkmtnt7000
Copy link
Member

Almost same results as python3

tsukamoto@tsukamoto-desktop ~/tmp/voicevox_ws 
$ pip freeze | grep setuptools
tsukamoto@tsukamoto-desktop ~/tmp/voicevox_ws 
$ dpkg -l | grep python-setuptools
ii  python-setuptools                               39.0.1-2                                        all          Python Distutils Enhancements

@iory
Copy link
Member Author

iory commented Apr 18, 2022

What is the result of python3 -c 'import setuptools; print(setuptools.__version__)'

@tkmtnt7000
Copy link
Member

tsukamoto@tsukamoto-desktop ~/tmp/voicevox_ws 
$ python3 -c 'import setuptools; print(setuptools.__version__)'
39.0.1

@Naoki-Hiraoka
Copy link
Contributor

Naoki-Hiraoka commented Apr 19, 2022

HRP2017で動作確認しました。今の所、問題なく使えそうです。 https://github.com/start-jsk/rtmros_hrp2/pull/607

@knorth55 knorth55 mentioned this pull request Sep 30, 2022
6 tasks
@knorth55
Copy link
Member

this PR should remove type annotation lines.
we cannot use this PR in PR2 + Indigo...

@a-ichikura
Copy link
Contributor

I tried to use voicevox with Ubuntu18.04 / Ros melodic, but I cannot build this package.

$ git log
commit fcd54f5f7ef28a87cbfe1f04d537693132e571fb (HEAD -> iory/voicevox)
Merge: 56747c3a cdef6543
Author: Aiko Ichikura <ichikura@jsk.imi.i.u-tokyo.ac.jp>
Date:   Tue Apr 2 16:02:12 2024 +0900

    Merge branch 'voicevox' of https://github.com/iory/jsk_3rdparty into iory/voicevox

The error message was too long, so I uploaded at this gist.

How should I do ?

@mqcmd196
Copy link
Member

mqcmd196 commented Apr 8, 2024

Hmm, I can build it in my Ubuntu 20.04 env. Let's check your PC

@k-okada
Copy link
Member

k-okada commented Apr 8, 2024

@mqcmd196 I have same error even on 20.04. https://gist.github.com/k-okada/0e69cc55b261f914de5c430e1290772a
can you shre your requirement.txt file?

@mqcmd196
Copy link
Member

mqcmd196 commented Apr 9, 2024

Here is the requirements.txt in noetic / Ubuntu 20.04 environment.

aiofiles==23.2.1          # via -r requirements.in
annotated-types==0.6.0    # via pydantic
anyio==4.3.0              # via starlette
appdirs==1.4.4            # via -r requirements.in
cffi==1.16.0              # via soundfile
click==8.1.7              # via uvicorn
cython==3.0.10            # via pyopenjtalk, pyworld
exceptiongroup==1.2.0     # via anyio
fastapi==0.110.1          # via -r requirements.in
h11==0.14.0               # via uvicorn
idna==3.6                 # via anyio
numpy==1.24.4             # via -r requirements.in, pyopenjtalk, pyworld, scipy
pycparser==2.22           # via cffi
pydantic-core==2.16.3     # via pydantic
pydantic==2.6.4           # via fastapi
git+https://github.com/VOICEVOX/pyopenjtalk@a85521a0a0f298f08d9e9b24987b3c77eb4aaff5#egg=pyopenjtalk  # via -r requirements.in
python-multipart==0.0.9   # via -r requirements.in
pyworld==0.3.4            # via -r requirements.in
pyyaml==6.0.1             # via -r requirements.in
scipy==1.10.1             # via -r requirements.in
six==1.16.0               # via pyopenjtalk
sniffio==1.3.1            # via anyio
soundfile==0.12.1         # via -r requirements.in
starlette==0.37.2         # via fastapi
tqdm==4.66.2              # via pyopenjtalk
typing-extensions==4.11.0  # via annotated-types, anyio, fastapi, pydantic, pydantic-core, starlette, uvicorn
uvicorn==0.29.0           # via -r requirements.in

Build log catkin b -j1 -p1 -v voicevox -> https://gist.github.com/mqcmd196/201af2a26f2612b402bb38ad1e539708

It seems that the error occurs in Ubuntu 18.04 environment PC borrowed from @a-ichikura . I'm handling it.

@k-okada
Copy link
Member

k-okada commented Apr 10, 2024

@mqcmd196 can you shere the output of

pip list -v | grep local
pip list -v | egrep -e '^(pip|wheel|setup)' 

@mqcmd196
Copy link
Member

pip -V
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
❯ pip list -v | grep local
absl-py                              1.4.0                /usr/local/lib/python3.8/dist-packages           pip
aiohttp                              3.8.4                /home/obinata/.local/lib/python3.8/site-packages pip
aiosignal                            1.3.1                /home/obinata/.local/lib/python3.8/site-packages pip
anyio                                3.7.1                /home/obinata/.local/lib/python3.8/site-packages pip
async-timeout                        4.0.2                /home/obinata/.local/lib/python3.8/site-packages pip
base-local-planner                   1.17.3               /opt/ros/noetic/lib/python3/dist-packages
black                                23.9.1               /usr/local/lib/python3.8/dist-packages           pip
cachetools                           5.2.0                /usr/local/lib/python3.8/dist-packages           pip
chainer                              7.8.1                /usr/local/lib/python3.8/dist-packages           pip
click                                8.1.7                /usr/local/lib/python3.8/dist-packages           pip
cmake-language-server                0.1.6                /usr/local/lib/python3.8/dist-packages           pip
datasets                             2.14.5               /home/obinata/.local/lib/python3.8/site-packages pip
debugpy                              1.8.0                /usr/local/lib/python3.8/dist-packages           pip
dill                                 0.3.7                /home/obinata/.local/lib/python3.8/site-packages pip
distro                               1.8.0                /home/obinata/.local/lib/python3.8/site-packages pip
dlib                                 19.24.0              /usr/local/lib/python3.8/dist-packages           pip
exceptiongroup                       1.2.0                /home/obinata/.local/lib/python3.8/site-packages pip
fcn                                  6.4.20               /usr/local/lib/python3.8/dist-packages           pip
freezegun                            1.2.1                /usr/local/lib/python3.8/dist-packages           pip
frozenlist                           1.3.3                /home/obinata/.local/lib/python3.8/site-packages pip
fsspec                               2023.6.0             /home/obinata/.local/lib/python3.8/site-packages pip
fysom                                2.1.6                /usr/local/lib/python3.8/dist-packages           pip
gdown                                4.5.1                /usr/local/lib/python3.8/dist-packages           pip
google-api-core                      2.8.2                /usr/local/lib/python3.8/dist-packages           pip
google-api-python-client             2.55.0               /usr/local/lib/python3.8/dist-packages           pip
google-auth                          2.9.1                /usr/local/lib/python3.8/dist-packages           pip
google-auth-httplib2                 0.1.0                /usr/local/lib/python3.8/dist-packages           pip
google-cloud-pubsub                  2.13.4               /usr/local/lib/python3.8/dist-packages           pip
google-cloud-texttospeech            2.12.0               /usr/local/lib/python3.8/dist-packages           pip
googleapis-common-protos             1.56.4               /usr/local/lib/python3.8/dist-packages           pip
grpc-google-iam-v1                   0.12.4               /usr/local/lib/python3.8/dist-packages           pip
grpcio                               1.47.0               /usr/local/lib/python3.8/dist-packages           pip
grpcio-status                        1.47.0               /usr/local/lib/python3.8/dist-packages           pip
h11                                  0.14.0               /home/obinata/.local/lib/python3.8/site-packages pip
httpcore                             1.0.2                /home/obinata/.local/lib/python3.8/site-packages pip
httplib2                             0.20.4               /usr/local/lib/python3.8/dist-packages           pip
httpx                                0.25.2               /home/obinata/.local/lib/python3.8/site-packages pip
huggingface-hub                      0.17.2               /home/obinata/.local/lib/python3.8/site-packages pip
libpgm                               1.3                  /usr/local/lib/python3.8/dist-packages           pip
multidict                            6.0.4                /home/obinata/.local/lib/python3.8/site-packages pip
multiprocess                         0.70.15              /home/obinata/.local/lib/python3.8/site-packages pip
mypy-extensions                      1.0.0                /usr/local/lib/python3.8/dist-packages           pip
nodeenv                              1.7.0                /usr/local/lib/python3.8/dist-packages           pip
oauth2client                         4.1.3                /usr/local/lib/python3.8/dist-packages           pip
openai                               1.3.5                /home/obinata/.local/lib/python3.8/site-packages pip
packaging                            23.1                 /usr/local/lib/python3.8/dist-packages           pip
pathspec                             0.11.2               /usr/local/lib/python3.8/dist-packages           pip
Pillow                               9.2.0                /usr/local/lib/python3.8/dist-packages           pip
pixel-ring                           0.1.0                /usr/local/lib/python3.8/dist-packages           pip
platformdirs                         3.10.0               /usr/local/lib/python3.8/dist-packages           pip
proto-plus                           1.20.6               /usr/local/lib/python3.8/dist-packages           pip
protobuf                             3.20.1               /usr/local/lib/python3.8/dist-packages           pip
pyarrow                              13.0.0               /home/obinata/.local/lib/python3.8/site-packages pip
pydantic                             1.10.5               /usr/local/lib/python3.8/dist-packages           pip
PyDrive                              1.3.1                /usr/local/lib/python3.8/dist-packages           pip
pygithub3                            0.5.1                /usr/local/lib/python3.8/dist-packages           pip
pygls                                0.12.4               /usr/local/lib/python3.8/dist-packages           pip
pyright                              1.1.294              /usr/local/lib/python3.8/dist-packages           pip
PySocks                              1.7.1                /usr/local/lib/python3.8/dist-packages           pip
pytesseract                          0.3.9                /usr/local/lib/python3.8/dist-packages           pip
pyusb                                1.2.1                /usr/local/lib/python3.8/dist-packages           pip
rsa                                  4.9                  /usr/local/lib/python3.8/dist-packages           pip
slacker                              0.13.0               /usr/local/lib/python3.8/dist-packages           pip
slacker-cli                          0.4.2                /usr/local/lib/python3.8/dist-packages           pip
sniffio                              1.3.0                /home/obinata/.local/lib/python3.8/site-packages pip
SpeechRecognition                    3.8.1                /usr/local/lib/python3.8/dist-packages           pip
spidev                               3.5                  /usr/local/lib/python3.8/dist-packages           pip
tabulate                             0.8.10               /usr/local/lib/python3.8/dist-packages           pip
tomli                                2.0.1                /usr/local/lib/python3.8/dist-packages           pip
tqdm                                 4.64.0               /usr/local/lib/python3.8/dist-packages           pip
typeguard                            2.13.3               /usr/local/lib/python3.8/dist-packages           pip
typing-extensions                    4.8.0                /home/obinata/.local/lib/python3.8/site-packages pip
uritemplate                          4.1.1                /usr/local/lib/python3.8/dist-packages           pip
webrtcvad                            2.0.10               /usr/local/lib/python3.8/dist-packages           pip
xxhash                               3.3.0                /home/obinata/.local/lib/python3.8/site-packages pip
yarl                                 1.9.2                /home/obinata/.local/lib/python3.8/site-packages pip
❯ pip list -v | egrep -e '^(pip|wheel|setup)'
pip                                  20.0.2               /usr/lib/python3/dist-packages
setuptools                           45.2.0               /usr/lib/python3/dist-packages
wheel                                0.34.2               /usr/lib/python3/dist-packages

@k-okada
Copy link
Member

k-okada commented Apr 10, 2024

how about

$ dpkg --get-selections | grep cython
cython						install
cython3						install
$ which cython
/usr/bin/cython
$ which cython3
/usr/bin/cython3
$ rosversion catkin_virtualenv
0.6.1
$ rospack find catkin_virtualenv
/opt/ros/noetic/share/catkin_virtualenv

@mqcmd196
Copy link
Member

❯ dpkg --get-selections | grep cython
cython3	
❯ which cython
cython not found
❯ which cython3
/usr/bin/cython3
❯ rosversion catkin_virtualenv
0.6.1
❯ rospack find catkin_virtualenv
/opt/ros/noetic/share/catkin_virtualenv

@mqcmd196
Copy link
Member

mqcmd196 commented Apr 12, 2024

@k-okada @iory
Could you try iory#10 ?

I succeeded the build in both @a-ichikura 's melodic environment and my noetic environment

@mqcmd196
Copy link
Member

mqcmd196 commented Apr 12, 2024

@k-okada @a-ichikura
I updated iory#10 . Forcely downgrade numpy version in my repo https://github.com/mqcmd196/pyopenjtalk/tree/support-old-numpy .

NOTE: the voicevox doesn't execute correctly in latest catkin_virtualenv for failing to get path in https://github.com/iory/jsk_3rdparty/blob/cdef6543059de03460e338039e07c2aed8714fbd/3rdparty/voicevox/node_scripts/server.py#L35 . Using apt-version of catkin_virtualenv works well

[VOICEVOX] Upgrade pyopenjtalk and lock versions in requirements
@a-ichikura
Copy link
Contributor

I launch voicevox_texttospeech.launch with setting launch_sound_play param to be true, however, the python program cannot find sound_play node.

$ roslaunch voicevox_texttospeech.launch launch_sound_play:=true --screen
... logging to /home/ichikura/.ros/log/672a4b7c-0e93-11ef-978a-287fcf9948c3/roslaunch-ichikura-ThinkPad-T495s-13602.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
WARNING: disk usage in log directory [/home/ichikura/.ros/log] is over 1GB.
It's recommended that you use the 'rosclean' command.

started roslaunch server http://ichikura-ThinkPad-T495s:34597/

SUMMARY
========

PARAMETERS
 * /rosdistro: melodic
 * /rosversion: 1.14.13
 * /sound_play_jp/default_voice: 2
 * /voicevox_server/cpu_num_threads: 1

NODES
  /
    sound_play_jp (sound_play/soundplay_node.py)
    voicevox_server (voicevox/server.py)

auto-starting new master
process[master]: started with pid [13623]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 672a4b7c-0e93-11ef-978a-287fcf9948c3
process[rosout-1]: started with pid [13641]
started core service [/rosout]
/opt/ros/melodic/lib/python2.7/dist-packages/roslib/packages.py:470: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if resource_name in files:
process[voicevox_server-2]: started with pid [13648]
process[sound_play_jp-3]: started with pid [13649]
INFO:     Started server process [13648]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:50021 (Press CTRL+C to quit)
[INFO] [1715321175.606535]: Loading from plugin definitions
[INFO] [1715321176.113933]: sound_play node is ready to play sound

From this messages, I understand sound_play node is working and I can see sound_play_jp node in an another terminal with rosnode list.

$ rosnode list
/rosout
/sound_play_jp
/voicevox_server

However, I got error messages which says soundplay_node.py is not working.

$ python3 voicevox_test.py 
[WARN] [1715321499.986613]: Sound command issued, but no node is subscribed to the topic. Perhaps you forgot to run soundplay_node.py?

The content of the python file is just simple example.

#!/usr/bin/env python3                                                                               
# -*- coding: utf-8 -*-                                                                              

import rospy
from sound_play.libsoundplay import SoundClient

rospy.init_node('say_node')

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp')

client.say('こんにちは', voice='四国めたん-あまあま')

How can I solve it?

@mqcmd196
Copy link
Member

I encountered same problem.

Adding sleep worked well like

#!/usr/bin/env python3

import rospy
from sound_play.libsoundplay import SoundClient

rospy.init_node("say_node")

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp')

rospy.sleep(2)

client.say('こんにちは', voice='四国めたん-あまあま')

is it sound_play issue?

@k-okada
Copy link
Member

k-okada commented May 11, 2024

Or, you can use actionlib version

#!/usr/bin/env python3                                                                               
# -*- coding: utf-8 -*-                                                                              

import rospy
from sound_play.libsoundplay import SoundClient

rospy.init_node('say_node')

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp', blocking=True)

client.say('こんにちは', voice='四国めたん-あまあま')

https://github.com/ros-drivers/audio_common/blob/6baeb0739be642286fc3ceb3797a49906ba4c64b/sound_play/src/sound_play/libsoundplay.py#L342-L368

@mqcmd196
Copy link
Member

mqcmd196 commented May 11, 2024

btw, I encounter these error both my desktop and laptop environment when the sound play request has been sent

❯ roslaunch voicevox voicevox_texttospeech.launch
... logging to /home/obinata/.ros/log/2d5e083e-0f38-11ef-a315-154c2f26852b/roslaunch-sinobuz-18211.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
WARNING: disk usage in log directory [/home/obinata/.ros/log] is over 1GB.
It's recommended that you use the 'rosclean' command.

started roslaunch server http://sinobuz:36005/

SUMMARY
========

PARAMETERS
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /sound_play_jp/default_voice: 2
 * /voicevox_server/cpu_num_threads: 1

NODES
  /
    sound_play_jp (sound_play/soundplay_node.py)
    voicevox_server (voicevox/server.py)

auto-starting new master
process[master]: started with pid [18219]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 2d5e083e-0f38-11ef-a315-154c2f26852b
process[rosout-1]: started with pid [18229]
started core service [/rosout]
process[voicevox_server-2]: started with pid [18236]
process[sound_play_jp-3]: started with pid [18237]
/home/obinata/ros/review_ws/src/jsk_3rdparty/3rdparty/voicevox/node_scripts/server.py:9: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/opt/ros/noetic/lib/sound_play/soundplay_node.py:330: PyGIDeprecationWarning: Since version 3.11, calling threads_init is no longer needed. See: https://wiki.gnome.org/PyGObject/Threading
  GObject.threads_init()
/opt/ros/noetic/lib/sound_play/soundplay_node.py:331: PyGIDeprecationWarning: GObject.MainLoop is deprecated; use GLib.MainLoop instead
  self.g_loop = threading.Thread(target=GObject.MainLoop().run)
/home/obinata/ros/review_ws/src/jsk_3rdparty/3rdparty/voicevox/node_scripts/server.py:109: DeprecationWarning:
        on_event is deprecated, use lifespan event handlers instead.

        Read more about it in the
        [FastAPI docs for Lifespan Events](https://fastapi.tiangolo.com/advanced/events/).

  @app.on_event("startup")
INFO:     Started server process [18236]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:50021 (Press CTRL+C to quit)
[INFO][1715391944.356065][/sound_play_jp:rosout]: Loading from plugin definitions
[INFO][1715391944.860853][/sound_play_jp:rosout]: sound_play node is ready to play sound
[Text2Wave] Speak using voice_name (四国めたん-あまあま)..
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/fastapi/routing.py", line 296, in app
    content = await serialize_response(
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/fastapi/routing.py", line 160, in serialize_response
    return field.serialize(
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/fastapi/_compat.py", line 147, in serialize
    return self._type_adapter.dump_python(
  File "/home/obinata/ros/review_ws/devel/.private/voicevox/share/voicevox/venv/lib/python3.8/site-packages/pydantic/type_adapter.py", line 333, in dump_python
    return self.serializer.to_python(
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'numpy.float32'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/obinata/ros/review_ws/src/jsk_3rdparty/3rdparty/voicevox/bin/text2wave", line 151, in <module>
    request_synthesis(speech_text,
  File "/home/obinata/ros/review_ws/src/jsk_3rdparty/3rdparty/voicevox/bin/text2wave", line 115, in request_synthesis
    rospy.logwarn("reponse: {}".format(response.json()))
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[ERROR][1715391962.703104][/sound_play_jp:rosout]: Sound synthesis failed.Is festival installed?Is a festival voice installed?Try running "rosdep satisfy sound_play|sh".Refer to http://wiki.ros.org/sound_play/Troubleshooting
[ERROR][1715391962.704263][/sound_play_jp:rosout]: Failed to generate wavfile.
[ERROR][1715391962.705175][/sound_play_jp:rosout]: Exception in actionlib callback: 'こんにちは---四国めたん-あまあま'
[INFO][1715391962.705970][/sound_play_jp:rosout]: Traceback (most recent call last):
  File "/opt/ros/noetic/lib/sound_play/soundplay_node.py", line 291, in execute_cb
    sound = self.select_sound(data)
  File "/opt/ros/noetic/lib/sound_play/soundplay_node.py", line 162, in select_sound
    sound = self.voicesounds[voice_key]
KeyError: 'こんにちは---四国めたん-あまあま'

@knorth55
Copy link
Member

@mqcmd196

#337 (comment)

this is a common issue for ros publisher.
after publisher initialization, we need to wait a bit to be ready for publishing.

@mqcmd196
Copy link
Member

mqcmd196 commented May 12, 2024 via email

@a-ichikura
Copy link
Contributor

@knorth55 @mqcmd196

Thank you for the information.
I succeeded in speaking the voice with 5 second sleep.

#!/usr/bin/env python3                                                                                                                      
# -*- coding: utf-8 -*-                                                                                                                     

import rospy
import time
from sound_play.libsoundplay import SoundClient

rospy.init_node('say_node')

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp')

time.sleep(5)

client.say('こんにちは', voice='四国めたん-あまあま')

@mqcmd196
Copy link
Member

mqcmd196 commented May 13, 2024 via email

@a-ichikura
Copy link
Contributor

Or, you can use actionlib version

#!/usr/bin/env python3                                                                               
# -*- coding: utf-8 -*-                                                                              

import rospy
from sound_play.libsoundplay import SoundClient

rospy.init_node('say_node')

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp', blocking=True)

client.say('こんにちは', voice='四国めたん-あまあま')

https://github.com/ros-drivers/audio_common/blob/6baeb0739be642286fc3ceb3797a49906ba4c64b/sound_play/src/sound_play/libsoundplay.py#L342-L368

This method is also useful.

@a-ichikura
Copy link
Contributor

@iory

How can I merge voicevox and is_speaking node?
I made itodenwa.launch as follow.

<launch>
  <arg name="language" default="JA-jp" />
  <arg name="launch_sound_play" default="false"/>

  <include file="$(find voicevox)/launch/voicevox_texttospeech.launch" />
    <group ns="robotsound" >
    <node name="is_speaking"
          pkg="sound_play" type="is_speaking.py" >
      <remap from="~robotsound" to="status" />
      <remap from="~output/is_speaking" to="is_speaking" />
    </node>
  </group>

  <group ns="robotsound_jp" >
    <node name="is_speaking"
          pkg="sound_play" type="is_speaking.py" >
      <remap from="~robotsound" to="status" />
      <remap from="~output/is_speaking" to="is_speaking" />
    </node>
  </group>

</launch>

The node is working well,

$ roslaunch itodenwa.launch 
... logging to /home/ichikura/.ros/log/be0431f8-1107-11ef-91dd-287fcf9948c3/roslaunch-ichikura-ThinkPad-T495s-27822.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ichikura-ThinkPad-T495s:34203/

SUMMARY
========

PARAMETERS
 * /rosdistro: melodic
 * /rosversion: 1.14.13
 * /sound_play_jp/default_voice: 2
 * /voicevox_server/cpu_num_threads: 1

NODES
  /
    sound_play_jp (sound_play/soundplay_node.py)
    voicevox_server (voicevox/server.py)
  /robotsound/
    is_speaking (sound_play/is_speaking.py)
  /robotsound_jp/
    is_speaking (sound_play/is_speaking.py)

ROS_MASTER_URI=http://localhost:11311

/opt/ros/melodic/lib/python2.7/dist-packages/roslib/packages.py:470: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if resource_name in files:
process[voicevox_server-1]: started with pid [27849]
process[sound_play_jp-2]: started with pid [27850]
process[robotsound/is_speaking-3]: started with pid [27851]
process[robotsound_jp/is_speaking-4]: started with pid [27852]
INFO:     Started server process [27849]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:50021 (Press CTRL+C to quit)
[INFO] [1715592317.112396]: Loading from plugin definitions
[INFO] [1715592317.621100]: sound_play node is ready to play sound

However, the topic data doesn't change(false to true) even thought the sound is playing.

…rver

This commit introduces a new utility function `convert_to_float` that recursively converts numpy.float32 values to Python's native float type. This change is necessary because numpy.float32 values are not serializable by Python's json library, which causes issues when these values need to be sent as JSON responses from the voicevox server.

The `convert_to_float` function is now applied to `accent_phrases` after they are generated by the engine, ensuring that all numeric values are in a format compatible with JSON serialization standards.

This update ensures that the server can handle serialization of voice properties without encountering errors due to data type compatibility issues with JSON.
@iory
Copy link
Member Author

iory commented May 13, 2024

It seems that your launch file is functioning correctly. Have you tried using
rostopic echo /robotsound_jp/is_speaking and calling the program below to see if the say function is called and True is displayed until the robot finishes speaking?

simple version

<launch>

  <include file="$(find voicevox)/launch/voicevox_texttospeech.launch" />

  <group ns="robotsound_jp" >
    <node name="is_speaking"
          pkg="sound_play" type="is_speaking.py" >
      <remap from="~robotsound" to="status" />
      <remap from="~output/is_speaking" to="is_speaking" />
    </node>
  </group>

</launch>
import rospy
from sound_play.libsoundplay import SoundClient

rospy.init_node('say_node')

client = SoundClient(sound_action='robotsound_jp', sound_topic='robotsound_jp', blocking=True)

client.say('こんにちは', voice='四国めたん-あまあま')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants